AI虛擬人多模態交互落地難題如何破解？我們在樂享A.I.技術沙龍成都站找到了答案

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"6月23日，訊飛開放平臺樂享A.I.技術沙龍“A.I.虛擬人多模態創新交互”專場在成都圓滿落幕。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"近幾年，隨着圖像處理、語音合成、語音識別、語義理解、多語種等多項人工智能技術不斷髮展，AI虛擬人開始在各行各業落地應用，各大公司爭相佈局。這背後，其實是AI虛擬人多模態交互技術正在成爲大趨勢。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"多模態融合視覺、聽覺、文本等多種模態信息，能夠打破單模態輸入輸出限制，從而更貼近人類真實使用習慣。對於AI交互產品來說，只有綜合利用多模態信息才能更準確地理解人類行爲。但現階段受限於語義鴻溝、異構鴻溝、數據缺失等原因，多模態研究還有一定難度，如何將多模態技術應用於實際產品仍然困擾着許多開發者。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"本次訊飛開放平臺樂享A.I.技術沙龍"},{"type":"text","text":"成都專場，致力於爲開發者們提供破解多模態技術落地難題的思路參考。活動現場吸引了衆多開發者參與，來自科大訊飛的多位分享嘉賓與成都的開發者夥伴齊聚一堂，圍繞“A.I.虛擬人多模態創新交互”主題，展開了面對面的深入探討與實操模擬。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/37\/37047c49d6a65c712150247f0eaad7c5.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"沙龍開場，"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"訊飛開放平臺總經理張斌"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"發表了開場致辭。他表示，成都是一個擁有巨大潛力和機會的城市，因此它對於很多企業來說有非常大的魅力。電子信息產業是成都的支柱產業，今年2月份成都剛剛成爲第二批國家人工智能應用先導區，這體現了成都在人工智能行業中的重要地位。在訊飛開放平臺上超過210萬開發者中，來自成都的開發者數量也居於各大城市前列。因此科大訊飛希望通過本場沙龍活動與成都的開發者們建立更好的聯繫，一起共建人工智能生態。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/fa\/fa342c7efd22ecaf879ac2f33f8ef7cb.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"語音合成領域發展趨勢深度洞察"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"如今，智能語音已經成爲新交互模態必不可少的技術之一，其中，語音合成能力讓產品走向“開口說話”的階段。在未來越來越多的個性化場景下，語音合成可能會往哪些方向發展？"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"訊飛AI研究院副院長江源"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在語音合成領域從事了十多年的研究，他以"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"“未來個性場景下語音合成發展趨勢”"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲主題，與現場開發者分享了他對於語音合成領域技術發展趨勢的深度洞察。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"江源指出，未來面對越來越豐富的個性化場景，語音合成技術的發展方向將主要由三個訴求牽引，分別是音色、情感和多語種。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/0d\/0d09b0a280559d95a3e92aa56002d058.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" 首先，大家都想要擁有一個獨一無二的聲音來代表自己、代表品牌形象，因此如何便捷高效地定製個性化音色是非常大的訴求。在這方面，科大訊飛已經能夠提供非常成熟的解決方案和產品，做到品牌個性化、應用個性化和用戶個性化，這背後主要依靠的是訊飛自研的基於聽感量化的混合數據多層級建模方案。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"其次是自如的情感表達，傳統的語音合成只要求做到信息播報，讓聽衆聽清就可以，現在隨着技術的進步，機器合成的聲音已經可以比較好地貼近真人的聲音，大家對於聲音和效果的要求也更多了，比如希望機器合成的聲音更好聽、更悅耳、更符合當前的使用場景，這就要求機器能夠更好地表達交互之間情感的細微變化。情感也是未來人機交互中很重要的特性，關係到人工智能是否可以跨越鴻溝實現真正的智能。目前科大訊飛正在圍繞可控情感語音合成技術和可控文本情感預測技術做一些研究和探索。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"最後是方言語種的多元需求，這主要是爲了幫助我們更好地保護地域性的語言和文化，以及更好地走向全球化。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"訊飛A.I.虛擬人方案及應用解讀"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"目前，AI主播、AI偶像等等AI虛擬人場景已經成爲各大公司重點佈局的方向，這一場景背後從視頻圖像到智能語音技術缺一不可。訊飛的AI虛擬人方案有哪些技術創新點？訊飛怎麼解決AI虛擬人在內容和交互上的問題？"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"訊飛有聲平臺業務部總經理郜靜文"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在以“"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"A.I.虛擬人內容生產和多模態交互”"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲主題的分享中，詳細介紹了訊飛A.I.虛擬人的方案及應用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"科大訊飛自成立以來就一直在從事A.I.虛擬人相關技術的研究，語音合成、語音識別、語義理解、圖像理解等技術都爲A.I.虛擬人多模態技術奠定了基礎。郜靜文在演講中分享了A.I.虛擬人應用的發展歷程，20世紀80年代，研究人員開始嘗試將虛擬人物引入到現實世界中，虛擬數字人步入萌芽階段；21世紀初，傳統手繪逐漸被CG、動作捕捉等技術取代，虛擬數字人步入探索階段；近五年，得益於深度學習算法的突破，虛擬數字人制作過程有效簡化，開始步入初級階段；互聯網、終端屏等技術的飛速發展和海量音視頻內容產出的需求，又進一步推動了A.I.虛擬人技術的發展，現如今，虛擬數字人正朝着智能化、便捷化、精細化、多樣化成長。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"訊飛A.I.虛擬人技術架構的最底層是基礎數據，包括業務知識和形象庫、聲音庫；往上一層是包括圖像識別、語義理解、3D人臉識別、語音驅動口脣、語音合成、語音識別、人臉視頻合成、肢體動作合成等在內的核心技術；再往上是基於核心技術打造出的2D真人、3D卡通、3D真人等虛擬形象。基於這些技術最後呈現給開發者的主要是三大類業務產品：播報系統，實現了文字到音頻以及文字到虛擬形象的輸出，也結合了相關的視頻處理文件的多軌能力，能夠支持輸入文字以後自動輸出需要的視頻，目前這套系統已經有一百多個媒體和企業在使用；多模態交互系統，可以應用在金融行業的視頻客服或視頻面試中；軟硬件一體的A.I.虛擬人一體化解決方案，可以放在展廳、大堂做一些交互和客服的工作，減少開發者和企業的開發成本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/ed\/ed0e26d639a1e487556e12319484dbb8.jpeg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" 郜靜文在現場發佈了科大訊飛A.I.虛擬人生態平臺，希望能有更多技術合作夥伴共同參與到虛擬人的研發和平臺建設中來。她表示，訊飛將在這個平臺中提供更多技術服務、SaaS服務和行業解決方案，與合作伙伴共建A.I.虛擬人生態圈。同時，訊飛對外開放2項定製能力，分別是A.I.音庫自訓練平臺和A.I.虛擬人服務調用，支持公有云調用、私有化部署。訊飛希望能與合作伙伴共同豐富虛擬人內容和服務，共創共贏。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"訊飛開放平臺產品矩陣"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"訊飛近幾年一直重點發力平臺+賽道的戰略，而平臺屬性對於生態建設來說必不可少，訊飛開放平臺也承載着訊飛在AI生態建設的野心。"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"訊飛開放平臺產品運營總監汪艦"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲現場開發者介紹了訊飛開放平臺目前可以提供哪些技術產品和解決方案，以及訊飛通過什麼樣的思路來優化這些產品。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/70\/70873fc69f33b49b2eb8307cda2fc8b2.jpeg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" 訊飛開放平臺成立於2010年，是基於科大訊飛國際領先的人工智能研究成果建設的AI技術與生態服務平臺，面向用戶提供語音識別、語音合成、語義理解、文字識別、人臉識別等AI技術授權。截至目前，訊飛開放平臺已經擁有330萬生態合作伙伴，對外開放了433項AI能力及解決方案，全球使用訊飛開放平臺技術能力的終端設備累計已經超過31億。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"汪艦表示，一項技術在實驗室誕生要走過的路，遠遠小於這項技術從實驗室走出來進入大家的生活場景要走的路。語音識別技術是訊飛的強項，但同樣也會收到很多客戶和使用者反饋的各種問題，爲了解決這些問題，訊飛研發團隊做了非常多努力。解決語音識別問題的核心是提高識別率，第一步是做好聲學前端處理，通過聲源定位、噪聲抑制、回聲消除等技術，從硬件上解決識別中的噪聲問題；第二步可以通過算法層面的聲學模型定製和語言模型定製來提高識別率；第三步可以藉助多模態技術，讓機器一邊聽一邊看，通過捕捉主要說話人來提高識別率。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"對於訊飛開放平臺的基本理念，汪艦也進行了解讀，他指出，開放平臺的源頭是數據，在很多場景下，數據來自各個行業的開發者和數據擁有者；拿到數據之後首先會進行數據標註，然後放到訊飛提供的機器學習引擎中去訓練，最終輸出一個引擎模型，訊飛會提供引擎部署服務，可以將這個引擎放到訊飛的開發平臺上做雲端的調用或本地化部署。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"除了精彩的大咖分享環節之外，沙龍最後還安排了"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"Workshop實操接入環節"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"，現場由"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"訊飛有聲平臺研發工程師呂磊"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"手把手現場教學，帶領開發者體驗訊飛AI虛擬人接入實踐，基於訊飛開放平臺能力，不同行業的開發者都能快速開發出可以滿足特定場景需求的A.I.虛擬人。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/5a\/5a9335412b4181e16b3f4be9dc382176.jpeg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"雖然成都專場活動已經落下帷幕，但訊飛開放平臺樂享A.I.技術沙龍系列活動才只是剛剛開始。接下來，訊飛開放平臺樂享A.I.技術沙龍系列活動將繼續向全國各大重點城市出發，圍繞AI熱點能力分享、行業應用探究和Workshop技術實踐等豐富形式，與各地開發者展開實地面對面交流，爲大家提供最新的人工智能理念、產品技術、解決方案和實踐經驗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"下一站，樂享A.I.技術沙龍將會走進哪個城市呢？讓我們拭目以待！"}]}]}