{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在人工智能技術不斷深入發展的今天，我們對於計算的性能要求越來越高。傳統的計算處理多數是基於雲側的，把所有圖像、音頻等數據通過網絡傳輸到雲中心進行處理後將結果反饋。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是隨着數據的指數式增長，依靠雲側的計算已經顯現了諸多不足，例如數據處理的實時性、網絡條件制約、數據安全等，因此端側的推理則愈發重要。在這樣的背景下，網易有道 AI 團隊自主設計研發了高性能端側機器學習計算庫——EMLL(Edge ML Library)，並已在近日開源。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"開源地址：https:\/\/github.com\/netease-youdao\/EMLL"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"EMLL 爲加速端側 AI 推理而設計，提供基於端側處理器的高性能機器學習計算庫，支持 fp32、fp16、int8 等數據類型，已在網易有道詞典筆、翻譯王和超級詞典等智能硬件產品的 NMT、ASR、OCR 引擎中應用，大幅提高計算性能，提升用戶體驗。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"端側AI"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"端側AI具有以下優勢："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"低延時"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"保證數據隱私"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不依賴網絡"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"端側AI挑戰："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"處理器算力有限，遠低於雲端計算能力，如何滿足日益複雜的端側AI性能的需求至關重要"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"內存大小和帶寬有限，對性能影響至關重要"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ARM 處理器在智能設備中占主導地位，是端側AI落地的主流平臺。NPU、DSP、GPU可以提供更高的計算能力，在端側AI上有一定的應用場景，但生態環境較差，距離成熟還需要時間。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"端側AI最耗時的計算爲全連接(FC)和卷積計算，底層核心計算爲矩陣乘，底層計算庫的性能對端側AI能否落地起決定性作用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"ARM第三方BLAS庫"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Eigen"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"線性代數運算的 C++ 模板庫，矩陣的運算可直接用符號做。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"OpenBLAS"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由中科院計算所維護的一個開源的高性能 BLAS 庫，基於Kazushige Goto 的 GotoBLAS，支持 Fortran BLAS 和 CBLAS 接口調用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"ARM Compute Library"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ARM 官方推出的計算庫，支持 AI 的常見運算，其中矩陣乘法運算以模型推理層的形式封裝，需要先初始化後才能調用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"表1 各ARM blas庫矩陣乘法特點"}]},{"type":"embedcomp","attrs":{"type":"table","data":{"content":"ARM blas庫 矩陣排列順序 對特定核心的指令優化 對扁平矩陣的優化 Eigen 支持任意行\/列主序 不充分 不充分 OpenBLAS 支持任意行\/列主序 對一部分核心如A53有彙編調優 除GEMV外沒有專門對扁平矩陣的優化 ARM Compute Library 默認行主序，列主序需另調轉置函數處理 對大部分核心均有彙編級別優化 權重矩陣固定時靠預重排提高效率，不固定時效率低"}}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"常規矩陣規模上的矩陣乘法進行了較好的優化，性能表現較好，然後在扁平矩陣上性能表現較差。端側AI底層計算主要爲扁平矩陣的乘法，第三方計算庫性能表現較差，沒有充分發揮硬件的性能，不利於AI應用在端側平臺上落地。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"表2 ARM cortex-A53四核第三方庫GEMM計算效率"}]},{"type":"embedcomp","attrs":{"type":"table","data":{"content":" 端側AI中部分矩陣乘法 Eigen OpenBLAS ARM Compute Library M = 128, N = 16000, K = 128 25% 36% 35% M = 7, N = 2048, K = 192 5% 6% 10% M = 23, N = 1536, K = 320 12% 10% 25%"}}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注：C(M, N) = A(M, K) * B(K, N)，以上值取全行主序和全列主序的最好值，測試在相同的矩陣上重複128次，計算效率由GEMM計算FLOPS值除以硬件理論FLOPS值得到。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"EMLL特點"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"高性能"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"EMLL實現的矩陣乘法函數，爲端側人工智能中常見的扁平矩陣的計算做了專門的優化，爲各常見ARM處理器做了特定的優化。對於cortex-A7\/A35\/A53\/A55\/A76處理器，本庫根據它們的流水線特點，使用了彙編級別的優化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"EMLL多數情況下相對Eigen、ARM compute Library第三方庫性能提升明顯，尤其在端側AI常用的扁平矩陣乘法中獲得數倍的性能提升。下圖展示了端側AI中部分典型矩陣尺寸情況下得單精度矩陣乘法的性能結果。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/61\/31\/614c675cd5e798260f14a467aa6d0731.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/50\/50c54a6e5b5093e343a8bdcd6072c044.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"圖1 EMLL矩陣乘法性能"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"易用性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"EMLL使用的函數接口在參數設計上力求簡潔直接，矩陣乘法去掉了不常用的LD*參數，矩陣和向量的傳遞通過指針和整數維度分別傳遞。本庫不依賴第三方計算庫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"擴展性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於矩陣乘法和量化函數，EMLL 庫提取了它們和架構無關的代碼作爲通用的宏，這些宏可以在支持新的CPU架構時大大節省所需的代碼量。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"EMLL性能優化方法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在端側設備上優化計算庫的性能，需要從訪存效率和計算效率兩個角度考慮，下面以（稠密）矩陣乘法爲例，介紹 EMLL 採用的優化方法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"分塊"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"矩陣乘法的計算過程中需要頻繁地訪存。當矩陣規模較大時，CPU 緩存容量不足以裝下其全部內容，訪存時就會頻繁出現緩存缺失，降低程序效率。此時，EMLL 會對矩陣乘法問題進行必要的拆解，把較大的矩陣切分成小塊的矩陣，這就是分塊的手段。經過切分，每個子任務只計算一小塊矩陣對結果的貢獻，只會密集訪問這個小塊矩陣的區域，大大提高了緩存命中率。對於兩個較大矩陣之間的乘法，EMLL 參照已有的優化工作[1]，通過多級的分塊，充分利用 CPU 多級緩存，主要採用如下兩種切分方式："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/50\/cd\/508d9daefc0d701424e68f83c84d3acd.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"圖2 分塊方法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"L1 - L3 代表不同矩陣塊所利用的 CPU 緩存"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CPU 的寄存器可以看成“速度最快的緩存”。爲了充分利用寄存器，在上述分塊的基礎上，EMLL 進行了進一步拆分，左邊的小矩陣拆成 m×k的最小矩陣 a1，右邊的小矩陣拆成 k×n 的最小矩陣 b1。計算這一對最小矩陣的乘法，如果直接用三重循環的方式，需要 2×m×n×k 次元素訪問，如果不利用寄存器，則都爲訪存操作；利用了寄存器，則只需要在乘法開始前把兩個小矩陣放到寄存器中，後續的乘法就不再訪存，使訪存減少到 (m + n) ×k 次。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"綜上，大尺度的分塊可以提高 CPU 各級緩存的利用率，小尺度的分塊可以利用 CPU 寄存器以減少訪存次數，兩者對性能均有明顯幫助。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"重排"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上文提到，爲了充分利用寄存器，子矩陣塊的讀取被劃分爲更小的小塊 m×k 或 k×n (1 < m, n, k < 20），計算中逐個讀取這些小塊。而通常情況下，矩陣在內存中的存儲方式爲行主序或列主序。無論哪種存儲方式，按小塊讀取會存在很多跳躍訪問的情況。跳躍訪問對性能不利，這裏列舉三點："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"消耗額外的緩存帶寬：L2\/L3 緩存與 L1 的數據交互以緩存行的形式進行。跳躍訪問 L2\/L3 緩存的數據時，緩存行數據的利用率低，浪費傳輸帶寬。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"無法充分利用向量化的加載單元：很多支持 SIMD 的 CPU 上配備了向量化的加載單元，支持一條指令加載若干個地址連續的元素，若是跳躍訪問則沒法利用此特性。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"增加頁表查詢的開銷：訪存操作常涉及虛擬地址向物理地址的轉換，需要查詢頁表。一個頁表的覆蓋地址範圍有限。如果跳躍的步長過大，則需頻繁查詢新的頁表。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在兩個子矩陣塊的乘法中，每個子矩陣塊通常會被讀取多次，每次讀取的順序可以相同。B 的子矩陣塊在與它相乘的 A 塊的行數多於 m時會被讀多次；A 的子矩陣塊在與它相乘的 B 塊的列數多於 n 時會被讀多次。EMLL 參照已有的優化工作1，在計算開始前，將兩個子矩陣塊先按計算時的讀取順序（即上一段所述按更小的小塊讀取）重新排列元素，這樣計算時對兩個子矩陣塊的訪問全部變成了順序訪問，此即重排的優化方法。雖然計算前重新排列元素會有額外的開銷，但計算過程中對矩陣塊的多次訪問被順序化後的收益更大，因此帶來總體的性能提升。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於特殊尺寸的矩陣，重排的開銷可能大於收益，需要選擇性地重排或不重排[2]。當源矩陣 A 的行數 M 很少而源矩陣 B 較大時，B 的子塊被重複讀取的次數大大降低，對 B 的子塊重排的收益大大降低，甚至開始低於開銷。這種情況在端側 AI 推理中非常普遍。EMLL 會判斷 M 的大小，當 M 小於一個閾值時，對矩陣 B 不再重排，而是調整計算順序，對 B 的所有元素進行順序的一次讀取。類似地，當源矩陣 B 的列數 N 明顯偏小時，EMLL 對矩陣 A 不再重排，調整計算順序，對 A 的元素一次順序讀完。通過對特殊尺寸矩陣的特別處理，EMLL 在這些尺寸上的性能明顯超過了 Eigen 和 OpenBLAS 等開源庫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"彙編優化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現今主流的 CPU 爲了提高數據計算的效率，支持了“單指令多數據”(SIMD)的處理模式，即一條指令對多個數據進行相同的操作。調用 SIMD 指令集，可以在不增加指令吞吐量的情況下，提高數據計算的吞吐量。ARM 平臺提供了 NEON 指令集支持 SIMD 操作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當 m = n = 4 而 k = 1 時，做最小矩陣小塊之間的乘法並累加結果，如果使用標量計算，需要 16 次乘法和 16 次加法。NEON 指令集提供了廣播模式的融合乘加操作，只需 4 條指令，即可完成相同的任務，如下圖所示。其他 m，n 和 k 的取值，大多也可以用 NEON 指令加速運算。NEON 指令可以通過彙編的方式顯式調用，也可通過編譯器提供的 intrinsics 函數調用，後者可讀性更好但性能指標的不確定性更大。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"端側的中低端平臺配備的處理器，爲了節省成本和功耗，在執行核心通常砍掉了亂序執行的能力，而是嚴格按照指令在指令流中的順序來執行它們，如 ARM 的 cortex-A7, A35, A53, A55 等。部分型號的處理器可以在順序執行的前提下同時執行相鄰的兩條指令。對於這些處理器，指令之間如果存在數據依賴或者執行單元衝突，則指令的順序會對性能產生明顯的影響，若追求極致性能，需要在彙編級別重排相關的指令。對於存在數據依賴的兩條指令（比如一個運算指令的輸入依賴於另一個加載指令的結果），應儘可能地使它們遠離，避免因依賴關係的等待造成流水線空閒。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"EMLL功能"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"支持的計算函數"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"表3 支持的計算函數"}]},{"type":"embedcomp","attrs":{"type":"table","data":{"content":" 計算函數 支持的數據類型 偏置 float32、int32 全連接fc float32 反量化 int32 -> float32 矩陣乘法 float32、float16、int8 重量化 int32 -> int16\/int8，int16 -> int8 量化 float32 -> int8\/int16"}}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"支持的架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"armv7a, armv8a"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"支持的端側操作系統"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Linux, Android"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"應用案例"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"網易有道詞典筆，是網易有道打磨的一款學習型智能硬件，網易有道詞典筆，具有“多行掃描翻譯”功能，支持整段翻譯的智能學習硬件。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"網易有道超級詞典打造高效的智能英語學習系統，強化端側功能，提供了拍照學英語、查詞翻譯、背單詞、聽力練習、對話翻譯、語音助手等功能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"網易有道翻譯王支持43種語言互譯，暢遊全球191個國家和地區，支持21種語言在線、7種語言端側拍照翻譯，指示牌、菜單等即拍即譯。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"網易有道詞典筆、超級詞典、翻譯王均內嵌了網易有道自主研發的神經網絡翻譯NMT、光學字符識別OCR、語音識別ASR、語音合成TTS等業內領先的AI技術，並且支持離線功能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"網易有道自研端側機器學習計算庫已在網易有道詞典筆、超級詞典、翻譯王等智能硬件產品中使用，帶來以下好處："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"端到端性能相對於使用eigen庫加速1.3到2.43倍，效果顯著，大大降低了端側推理引擎的延遲。除了在有道智能硬件帶來了較好的性能提升，我們在配置驍龍855的某款手機上也做了性能測試，端到端性能相對於eigen提升了25%-55%，效果明顯。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"端側推理引擎採用EMLL之後，可以上線更大的AI模型，提高質量，並保證實時性，如端側NMT質量(BLEU)提升2個點，端側ASR準確度提升4.73%。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"EMLL可以保證在更低端芯片上實時性，如在cortex-A7上使用Eigen庫無法達到實時性，使用EMLL之後延遲大幅降低，並保證實時性效果。EMLL可以讓智能硬件更多的芯片選擇，從而降低成本，提高市場競爭力。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"表4 測試平臺"}]},{"type":"embedcomp","attrs":{"type":"table","data":{"content":" 平臺 CPU型號 主頻(GHz) 有道詞典筆 A35 1.2 有道超級詞典 A53 1.5 有道翻譯王 A53 2.0 某手機(驍龍855) A76 2.8 RV1126 A7 1.5"}}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/a7\/a77dab81803a76b033f4ed364906e966.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖3 端側NMT、ASR、OCR在不同平臺上使用EMLL和eigen端到端性能加速比"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"EMLL 高性能端側機器學習計算庫，已經在網易有道多款智能硬件產品中實際應用並取得顯著的效果，大幅提升性能，給用戶帶來更好的產品體驗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"未來，網易有道將持續維護和優化EMLL，幫助更多企業、科研機構等夥伴提升端側AI計算能力。"}]}]} |