在元宇宙裏怎麼交朋友？Meta發佈跨語種交流語音模型，支持128種語言無障礙對話

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/www.infoq.cn\/article\/Zp1iOf6zrw5RuoRIcirI","title":"xxx","type":null},"content":[{"type":"text","text":"改名 Meta"}]},{"type":"text","text":" 之後，Facebook 的元宇宙願景正在一點點實現。這一次，Facebook 把目光投在了元宇宙社交上。"}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Meta 發佈語音處理模型 XLS-R"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"近日，Meta 正式發佈 "},{"type":"link","attrs":{"href":"https:\/\/ai.facebook.com\/blog\/xls-r-self-supervised-speech-processing-for-128-languages\/","title":"xxx","type":null},"content":[{"type":"text","text":"XLS-R"}]},{"type":"text","text":"——一套用於各類語音任務的新型自監督模型。據悉，XLS-R 由海量公共數據訓練而成（數據量是過去的十倍），能夠將傳統多語言模型的語言支持量增加兩倍以上。目前，XLS-R 共支持 128 種語言。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Meta 認爲，語音交流是人們最自然的一種交互形式。“隨着語音技術的發展，我們已經能夠通過對話同自己的設備及未來的虛擬世界直接互動，由此將虛擬體驗與現實世界融爲一體。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這與扎克伯格此前宣稱的“公司業務將以元宇宙優先”不謀而合。此前，扎克伯格曾概述了他建立“元世界”的計劃：一個建立在我們自己的數字世界之上的數字世界，包括虛擬現實和增強現實。“我們相信元宇宙將會接替移動互聯網”。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而 XLS-R 作爲元宇宙社交中必不可少的一環，可以幫助母語不同的人在元宇宙無障礙對話。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"值得一提的是，爲了通過單一模型實現對多種語言的廣泛語音理解能力，Meta 對 XLS-R 進行了微調，使其獲得語音識別、語音翻譯及語言識別等功能。據介紹，XLS-R 在 BABEL、CommonVoice 以及 VoxPopuli 語音識別基準測試，CoVoST-2 的外語到英文翻譯基準測試，以及 VoxLingua107 語言識別基準測試中都取得了不錯的成績。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了儘可能降低功能訪問門檻，目前，Meta 與 Hugging Face 聯手發佈了"},{"type":"link","attrs":{"href":"https:\/\/huggingface.co\/spaces\/facebook\/XLS-R-2B-22-16","title":"xxx","type":null},"content":[{"type":"text","text":"模型本體"}]},{"type":"text","text":"，並通過 fairseq GitHub repo 全面開放。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"XLS-R 工作原理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"據介紹，XLS-R 在 wav2vec 2.0 訓練集上接受了超過 43 萬 6 千小時的公開語音錄音訓練，從而實現了對語音表達的自監督學習方法。這樣的訓練量已經達到去年發佈的當時最強的模型 XLSR-53 的 10 倍。利用從會議記錄到有聲讀物的多種語音數據來源，XLS-R 的語言支持範圍擴展到 128 種，涵蓋的語種量達到前代模型的近 2.5 倍。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作爲 Meta 打造的有史以來最大模型，XLS-R 中包含超過 20 億個參數，性能遠高於其他同類模型。Meta 表示，事實證明，更多參數能夠更充分地體現、數據集中的各類語種。此外，Meta 還發現，規模更大的模型在單一語言預訓練方面的性能也同樣優於其他較小模型。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Meta 在四種主要多語言語音識別測試中對 XLS-R 做出評估，發現它在 37 種語言上獲得了超越以往模型的效能。具體測試場景爲：BABEL 中選取 5 種語言，CommonVoice 中選取 10 種語言，MLS 中選取 8 種語言，以及 VoxPopuli 上選取 14 種語言。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/resource\/image\/df\/71\/dfd599521b98056258740b65918b9771.png","alt":null,"title":"BABEL 上的單詞錯誤率基準測試結果。XLS-R 較前代模型實現了顯著改進。","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此外，Meta 還評估了語音翻譯模型，即將錄音資料直接翻譯成另一種語言。爲了打造一套能夠執行多種任務的模型， Meta 同時在 CoVoST-2 基準測試的數個不同翻譯方向上對 XLS-R 進行了微調，使其能夠在英語與多達 21 種語言之間實現內容互譯。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在使用 XLS-R 對英語以外的其他語言進行編碼時，獲得了顯著的效能提升，這也是多語言語音表達領域的一次重大突破。據 Meta 介紹，XLS-R 在低資源語言學習中實現了顯著改進，例如印尼語到英語的翻譯，其中 BLEU 準確率平均翻了一番。BLEU 指標的提升是指模型給出的自動翻譯結果與處理同一內容的人工翻譯結果間重合度更高，代表着模型在改進口語翻譯能力方面邁出了一大步。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/resource\/image\/f5\/1f\/f5fec2d6f9eec0993791c7d77yy7fc1f.png","alt":null,"title":"以 BLEU 指標衡量的自動語音翻譯準確率，其中較高值表示 XLS-R 從高資源語言（例如法語、德語）、中資源語言（例如俄語、葡萄牙語）或低資源語言（例如泰米爾語、土耳其語）語音記錄翻譯至英語時的準確率。","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Meta 認爲，XLS-R 證明擴大跨語言預訓練規模可以進一步提高低資源語言的理解性能。它不僅提高了語音識別率，同時也將由外語到英語的語音翻譯準確率提高了一倍以上。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“XLS-R 是我們朝着以單一模型理解多種不同語言（語音）目標邁出的重要一步，也代表着我們在利用公共數據推進多語言預訓練方面做出的最大努力。我們堅信這是一條正確的探索方向，將讓機器學習應用更好地理解所有人類語音、並促進後續研究，大大降低語音技術在全球範圍內、特別是服務匱乏社羣中的使用門檻。我們將不斷開發新方法，通過低監督學習拓展模型的語言理解能力、逐步使其覆蓋全球 7000 多種語言，實現算法的持續更新。”Meta 提到。"}]}]}