微軟圖靈通用語言表示模型 T-ULRv2 位居 XTREME 排行榜榜首

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"今天，我們很高興地宣佈，圖靈多語言模型（Turing multilingual language model，T-ULRv2）目前位居 Google "},{"type":"link","attrs":{"href":"https:\/\/sites.research.google\/xtreme","title":"","type":null},"content":[{"type":"text","text":"XTREME 公共排行榜"}]},{"type":"text","text":"榜首。該模型是由 Microsoft 圖靈團隊與 Microsoft Research 合作開發的，該模型的平均得分高出阿里巴巴（VECO）之前的最好成績 3.5 分。爲了實現這一點，除了預訓練模型之外，我們還使用了“StableTune”，這是一種基於穩定性訓練的新型多語言微調技術。排行榜上的其他模型包括 XLM-R、 mBERT、XLM 等。以前最好的提交之一也是來自 Microsoft 使用 "},{"type":"link","attrs":{"href":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/filter-an-enhanced-fusion-method-for-cross-lingual-language-understanding\/","title":"","type":null},"content":[{"type":"text","text":"FILTER"}]},{"type":"text","text":" 提交的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/ac\/4d\/ac28daee75ea9f445159a2af1508e44d.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"通用語言表示"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Microsoft 圖靈團隊長期以來，一直認爲語言表示應該是通用的。在這篇發表於 2018 年的"},{"type":"link","attrs":{"href":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/towards-language-agnostic-universal-representations\/","title":"","type":null},"content":[{"type":"text","text":"論文"}]},{"type":"text","text":" 中，我們提出了一種以無監督方式訓練語言不可知表示的方法。這種方法允許用一種語言對訓練過的模型進行微調，並以零樣本學習的方式應用於另一種語言。這將克服要求標籤數據以每種語言訓練模型的挑戰。自從這篇論文發表以來，無監督的預訓練語言建模已成爲所有自然語言處理模型的支柱，而基於 Transformer 的模型是所有這類創新的核心。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}