實體鏈接在OPPO小布助手和OGraph的實踐應用

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1 問題背景","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"精準直達的知識問答能力對營造小布“懂知識、更懂你”的形象非常重要。在語音助手場景，經常會出現一詞多義或者口語化表達等問題。例如:李白出裝，李白的詩，播放李白。第一李白是指遊戲角色，第二個是指詩人李白，第三個是指歌曲李白。如何精確識別用戶所指、給出正確的答案是小布助手面臨的挑戰。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/0a/0a719311bfa155609835e723508481b9.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"知識圖譜是讓機器理解客觀世界的基石,擁有極強的表達能力和建模靈活性。目前OPPO自建知識圖譜OGraph已經積累了數億實體和數十億關係。接下來讓我們看看小布和Ograph將如何碰撞，通過實體鏈接技術解決實體歧義問題，幫助小布做能聽、會說、更懂你的智能助手。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/11/114d16b3fca6285cf95224564ab473be.png","alt":null,"title":"圖：小布助手KBQA流程","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2 任務介紹","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實體鏈接是NLP、知識圖譜領域的基礎任務，即對於給定的一箇中文文本，將其中的實體描述（mention）與給定知識庫中對應的實體進行關聯的任務。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2009年在TAC會議上第一次提出實體鏈接任務，2014年深度學習流行之前都是通過統計特徵和基於圖的方法進行實體鏈接。2017年提出了Deep Joint方案，使用Attention結構進行語義匹配來實體消歧。後面該團隊通過模型結構創新，實現了同一個模型同時實體識別和消歧，消歧依舊使用Attention。2018年提出了Deep Type方案，將消歧問題轉化爲實體分類，得到實體類別後，通過Link Count確定要鏈接到的實體。2020年預訓練語言模型開始流行，Entity Knowledge方案藉助大量的語料和強大的預訓練模型，用序列標註的方式進行實體鏈接。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/aa/aae177db4f3f605ebe0c68f1de9efc56.png","alt":null,"title":"圖：實體鏈接發展歷程","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3 技術方案","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實體鏈接通常會被拆分爲三個子任務：實體識別、候選實體召回、實體消歧。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/44/444e7b6857dc6cf178a2feb839bb5244.png","alt":null,"title":"圖：實體鏈接流程示意\n","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.1 實體識別","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實體識別的作用是識別出query中的實體描述（即mention），例如李白所在朝代皇帝是誰中的李白即爲要識別的mention。在進行通用領域實體識別時，實體量級大、類型多、說法多樣，因此方案要兼顧效率和泛化。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"小布助手開發了基於詞典的實體識別使用自研匹配工具Word Parser，該工具無論是在性能還是功能都較開源工具有優勢。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/47/4785dc4d52931cec303d6621ed0e989c.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實體鏈接中並不關心實體類型，因此實體識別可以使用B/I標籤或者指針標註的模式，同時爲了提升輸入信息的豐富程度，引入詞彙信息作爲特徵補充，試驗了Lattice LSTM和FLAT等結構，實體識別效果提升約1%。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/27/27595436fa9c4d8bc07011104ef56fd5.png","alt":null,"title":"圖：FLAT示意","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.2 候選實體召回","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"候選實體召回的目標是用mention召回所有可能的候選實體，但是線上存在大量暱稱、同音字、口語化表達，例如：卷福、抖森這類暱稱別名，而且還會出現表述錯誤，例如豐滿奧特曼、拖拉機奧特曼、倩雅集，如果缺乏別名到正確實體的映射，就無法完成召回。在語音場景沒有用戶直接反饋，無法像搜索一樣通過用戶點擊進行挖掘，因此實體別名的挖掘相當有挑戰。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/bc/bcba494947d1662031fe988f2a67d888.png","alt":null,"title":"圖：實體別名示意\n","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據問題的特點，我們構建了兩套方案，通用別名挖掘採用信息抽取爲主模式生成爲輔的挖掘流程，充分利用OGraph中的描述信息、別名信息和關係信息，從中挖掘別名。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/50/5096e37fda2cf70b42d6e0370517ed11.png","alt":null,"title":"圖：通用別名挖掘流程","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對用戶輸入錯誤、同音字、近音字這類別名挖掘，小布助手開創性的構建了基於特徵聚類的別名發現流程。挖掘步驟如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1）Query篩選：採用領域關鍵詞過濾、搜索點擊日誌篩選的方式，從用戶搜索query和小布助手線上query中篩選出可能包含待挖掘別名的query。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6d/6d620215092e5d2c54cce5e75c74e743.png","alt":null,"title":"圖：錯誤別名示意","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2）實體識別：使用實體識別技術，從待挖掘query中識別實體。實體識別模型採用通用實體識別模型+垂域finetune的方式得到。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3）領域特徵構建：由於實體識別直接得到的實體別名準確率不高，而且沒有和實體標準名關聯起來，需要構建領域特徵，將挖掘的實體別名與實體標準名關聯。根據場景特點，我們選取了部首偏旁特徵和拼音特徵。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4）特徵聚類：使用聚類的方式來關聯挖掘的實體別名與實體標準名。通過使用這套挖掘方案，得到數十萬實體別名，準確率95%+，解決了線上高頻的別名問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/44/44dd6e38f485dbbebeb38e56faf5b029.png","alt":null,"title":"圖：基於特徵聚類的別名發現流程","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.3 實體消歧","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實體消歧是實體鏈接中最關鍵一步，消歧的本質是排序，通過對候選實體進行排序，得到每個候選實體的排序分數，選出最終的結果。實體消歧有下面主要問題：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1）在語音助手場景下多爲短文本，因此上下文特徵缺失，需要額外構造特徵幫助消歧。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2）消歧時不能僅僅通過語義特徵來消歧，還要充分考慮全局消歧特徵，例如“劉德華給學員講課，把炸彈比喻成姑娘的手”中的劉德華，從語義上來講清華大學教授劉德華更符合，實際上是演員劉德華和《拆彈專家》的文章標題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/1c/1c902fe89ddf483b55f40252e3752636.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3）圖譜中存在未對齊實體，導致模型消歧困難，同時容易出現語料標註錯誤。例如中國和中華人民共和國在有些開源圖譜中是兩個實體，導致訓練集裏有些正確標籤中國這個實體，有的正確標籤爲中華人民共和國這個實體。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對上面提到的問題，我們從數據準備、模型選型、模型優化等方面着手，通過針對性的優化解決這些問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.3.1 數據準備","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"構造消歧樣本時，我們提供儘可能多的信息給模型，樣本由三部分構成：query樣本、實體樣本和統計特徵。   ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"query樣本構造：query樣本輸入時關鍵時將mention的位置信息傳入模型，讓模型能判斷mention在query中的具體位置。因此引入標識符#將，在mention兩邊加入統一的標識符“#”，樣本如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/03/0395dcc1300f4180d7cec9d8f2b77a57.png","alt":null,"title":"圖：query樣本示意","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實體描述樣本構造：實體樣本中需要包含待消歧實體的特徵，我們在第一版實體樣本中設計了mention和標準名的拼接表示標準名和mention是否相同這一特徵；構造了“類型：實體類型”這種描述，提供實體類型信息；同時加入了實體描述和圖譜的三元組信息。在第二版中，我們將標準名和mention是否相同的特徵直接轉化成文本描述“名稱相同/不同”,同時加入了“非/是主義項”這一特徵，通過優化特徵表達方式，模型表現提升約2%。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/be/be993eec214576be565cf4049c1e6c40.png","alt":null,"title":"圖：實體樣本示意","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"統計特徵樣本構造：爲了避免消歧模型只關注語義特徵，我們統計了query和實體樣本共現特徵、流行度、豐富度、mention共現特徵等統計特徵，作爲模型輸入，輔助進行全局消歧。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d4/d43522eddb1023b24fb7c3a2ca36b044.png","alt":null,"title":"圖：統計特徵示意","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.3.2 模型選型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"排序學習中，有三種常見模式pointwise，pairwise和listwise，對於實體消歧這種只需要TOP1的排序任務，並不需要考慮候選實體之間的順關係，只考慮全局相關性，因此我們選取了pointwise方法。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e7/e72f52be8a128731ef5ff7e5b7549f07.png","alt":null,"title":"圖：排序學習模型示意","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"總結前人的消歧模型，Deep Joint從排序出發，Deep Type從分類出發,都取得了較好的效果，說明分類和排序任務都對消歧有幫助，因此我們根據任務特點，設計了多任務模型框架，同時進行排序和分類，兩個任務共享模型參數，一起訓練，損失函數一起優化，通過共享排序任務和分類任務的信息，模型可以有更好的表現，多任務損失函數如下。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/43/43274566ac6d3dd9f29197ca5e359fe0.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了更好的融入統計特徵，我們直接將統計特徵embedding拼接到特徵向量後面進行排序。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/df/dfd303406977363e800fd2ab0ac3ff44.png","alt":null,"title":"圖：實體消歧模型示意","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最終我們模型結構如下，將query樣本和實體樣本拼接，輸入預訓練語言模型，將CLS位置的向量和統計特徵embedding拼接後作爲特徵向量。排序任務將特徵向量輸入全連接層，然後經過tanh最終輸出[-1,1]區間的分數，分數越高代表越有可能是目標實體。分類任務將特徵向量輸入全鏈接層，經過softmax層輸出各個分類的得分。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過試驗基於多任務的實體鏈接模型效果整體上優於單任務模型，具體F1見下表。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/09/0999a5b2dca270f711299f41d1ffbaa9.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.3.3 模型優化","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了瞭解模型到底關注哪些輸入特徵，使用互信息的可視化方法對模型進行可視化，對各個詞的重要程度進行了可視化，顏色越深重要程度越高。通過可視化發現，模型傾向於關注實體的類型和用來區分實體的片段，例如示例1喫得慢、食物、喫法、火腿腸，示例2中的珊迪、海綿寶寶、開關電源品牌。示例3種的人物、種族、夢三國等，可以看到多任務模型關注的特徵都是對消歧有幫助的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/68/685603ed8f28723dd7078e37e7cab0c1.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/24/249d11286eef14de76d1c501c05f47f2.png","alt":null,"title":"圖：模型可視化","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"置信學習（Confident Learning，CL）是一種識別標籤錯誤、表徵標籤噪聲的算法框架。針對標註錯誤的問題根據置信學習的思想，在原始數據上用n-flod方式訓練了5個模型，用這些模型預測原始訓練集的標籤，然後融合5個模型輸出的標籤作爲真實標籤，再從原始訓練集中清理真實標籤與原標籤不一致的樣本，我們清理的樣本佔比3%，其中標籤錯誤的80%+。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ec/ec6e433d418fdf9c50e6f297303c9af0.png","alt":null,"title":"圖：對抗訓練流程示意","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對抗訓練是指在模型的訓練過程中構建對抗樣本，參與模型訓練的方法。正常訓練過程中，如果梯度方向陡峭，那麼很小的擾動都會產生很大的影響。爲了防止這種擾動，對抗訓練在模型訓練的過程中使用帶擾動的對抗樣本進行攻擊，從而提升模型的魯棒性。可以看到FGM和PGD兩種生成對抗樣本的方式效果提升都比較明顯。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/98/98dee1e0cec9c0d0b6384522e5847777.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"4 技術應用","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.1 小布助手中的應用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3b/3bab6f43eb2b9ca0591ad7d8bca7d4e3.png","alt":null,"title":"圖：小布助手KBQA流程示意","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在小布助手場景中，用戶對語音助手的智能程度抱有很大的期望，會問各類有趣的問題，比如多跳問題、六度關係查詢等。藉助實體鏈接技術小布助手可以精確識別用戶所指，配合子圖匹配技術可以處理實體問答、結構化問答、多跳問答、六度關係查詢等，覆蓋大部分結構化問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/38/381c3453b0678e1d5a57350ab3eca9fb.png","alt":null,"title":"圖：小布助手各類結構化問答示意","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了驗證小布助手的實體鏈接能力，我們在自建評測集和千言評測集上都測試了我們的算法，可以取得良好的效果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/98/98dee1e0cec9c0d0b6384522e5847777.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.2 OGraph中的應用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實體鏈接不僅可以應用到KBQA，在信息抽取流程中也是至關重要的一環。Ograph信息抽取流程使用了信息抽取模型Casrel和專門針對實體類問題訓練的MRC模型，可以獲得海量候選三元組。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"得到三元組後需要利用實體鏈接技術，將三元組與知識庫中的實體鏈接起來。爲了保證實體鏈接的召回，我們優化了候選實體召回能力，同時使用挖掘的實體別名詞表和ES檢索系統進行召回，確保鏈接效果，通過整個流程生產三元組數百萬。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d6/d6b6c557ed9e7bb77a2913d272fcaecf.png","alt":null,"title":"圖：OGraph信息抽取流程","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"5 總結","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過在實體鏈接技術上的探索，小布助手可以更好的解析各類實體口語化表達、別名和歧義，藉助KBQA其他流程和OGraph豐富的知識儲備，已經能夠解答大部分用戶提出的問題，自然語言理解道阻且長，小布助手的進化永遠不會停止。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"6 參考文獻","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1. Deep Joint Entity Disambiguation with Local Neural Attention. Octavian-Eugen Ganea, Thomas Hofmann.","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2.Improving Entity Linking by Modeling Latent Entity Type Information,Shuang Chen, Jinpeng Wang, Feng Jiang, Chin-Yew Lin.","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3.End-to-End Neural Entity Linking. Nikolaos Kolitsas, Octavian-Eugen Ganea, Thomas Hofmann.","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4. Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking, Samuel Broscheit.","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"5.Towards Deep Learning Models Resistant to Adversarial Attacks. A Madry, A Makelov, L Schmidt, D Tsipras.","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"6.Confident Learning: Estimating Uncertainty in Dataset Labels. Curtis G. Northcutt, Lu Jiang, Isaac L. Chuang.","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"7.Towards a Deep and Unified Understanding of Deep Neural Models in NLP. Chaoyu Guan, Xiting Wang, Quanshi Zhang, Runjin Chen, Di He, Xing Xie.","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"作者簡介","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"FrankFan  ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"OPPO高級NLP算法工程師","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"主要從事對話和知識圖譜相關工作，在實體挖掘、序列標註、關係抽取、實體鏈接等方向具有豐富的研發經驗，積累專利十餘篇。","attrs":{}}]}]}