移動AI系列-百度App AI統一接入方案-MML

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一、背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，越來越多的AI場景將AI能力直接部署在移動端，其優勢主要在於實時、省流、以及安全性等方面，也稱爲端側AI。端側AI給移動端產品帶來巨大的想象空間，促進了移動互聯網下半場的繁榮。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然而在移動端直接部署AI，成本相對較高。首先需要選擇開源的移動端預測引擎，並將模型轉換爲該引擎所支持的格式，如百度支持多平臺的高性能預測引擎——Paddle-Lite。雖然預測引擎已經儘量設計的簡單易用，並且對移動端開發者友好，但還是有一定的學習成本；預測引擎集成完畢，業務開發中還需要考慮模型資源的下發及加載，輸入的前處理，輸出的後處理，資源釋放，模型升級，模型加解密等諸多環節。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外，有些特殊業務場景可能還會用到不同的預測引擎，此時若面向不同預測引擎的API進行開發，將會增加業務開發和維護成本。在此背景下，也是在百度App多次移動AI部署的經驗總結中，MML應運而生。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"二、MML架構設計"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MML（Mobile Machine Learning）作爲移動端AI統一接入方案， 其架構設計如下圖："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/2e\/13\/2ec8bdd764db718f016d95e6749b5513.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最底層是移動端預測引擎，這裏也稱爲Backend，例如百度的深度學習引擎Paddle-Lite，Paddle-Mobile（Paddle-Lite的前身），百度的機器學習引擎BML，以及蘋果的CoreML等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"往上則是MML，是處於底層預測引擎與上層業務之間的隔離層。對於上層業務，其屏蔽了底層不同預測引擎的細節，使上層業務面向統一的接口開發，並提供多種基礎能力來提高開發效率。例如百度App目前正在將預測引擎從Paddle-Mobile升級到Paddle-Lite，如果每項AI業務之前都是通過MML接入的Paddle-Mobile，那麼升級到Paddle-Lite將非常容易，接口調用上幾乎沒有差別，只要指明所使用的預測引擎即可，後面我們可以看到MML是如何抽象這些細節。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再往上則是業務層，如百度App中的Lens，圖片\/視頻超分辨率等AI能力。另外，MML也會逐步對一些通用AI能力進行封裝，如超分辨率，手勢檢測等，在未來隨MML一起開源，進一步幫助開發者降低移動端AI部署成本。所以MML本身可以分爲兩層，包括基礎層MML Core，以及通用AI能力層MML Kit。最上層則是宿主App"},{"type":"text","marks":[{"type":"strong"}],"text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"三、MML Core"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MML Core作爲基礎層，對所支持的多款預測引擎的Api進行了抽象封裝，使得開發者可以面向統一接口編程，業務代碼與預測引擎的細節隔離開來。目前，預測引擎的推理過程基本都可以抽象爲初始化，前處理，推理，後處理，釋放資源五個步驟。因此，不管開發者使用何種預測引擎，只要是MML所支持的，都可以通過MML的抽象接口進行開發，只要指定所用Backend即可，添加或切換新的Backend將會非常方便。同時，MML不僅提供Java和OC接口給移動端開發者使用，還提供了C++接口，這樣不僅降低性能損耗，還可以雙端共用一套代碼。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MML Core除了對預測API的抽象封裝，還提供了多種基礎能力和工具集，包括模型解密，前後處理工具集，以及任務調度，profiler統計等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1. 工具集"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.1 模型解密"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"模型的安全是移動端AI部署的一個重要問題，MML集成了百度自研的模型加解密算法，可以在內存中對加密模型進行解密。另外，業務方也可以選擇自己解密，或者交給Backend解密。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.2 前後處理工具集"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深度學習模型，對輸入數據有各種各樣的要求，如數據類型，數據排布，以及尺寸等等，輸出數據同樣也需要進行相應處理才能交給上層使用。這些操作如果都使用OpenCV來實現，性能可能達不到上線要求，MML前後處理工具集對多類OpenCV函數進行了彙編級優化，避免前後處理成爲性能瓶頸。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.3 任務調度"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"任務調度是MML目前仍在完善的功能。一個模型在某款移動設備上運行，在CPU上更快，還是在GPU上更快，目前採用的動態測速方式會進一步升級，根據模型計算量以及設備硬件信息來推斷和決定運行在哪裏，以及更進一步地對模型進行混合調度，使部分算子運行在CPU，部分算子運行在GPU。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":" 2. 接下來，我們看一下使用MML進行預測的各個步驟："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/73\/71\/736c3cf634530c4bd76ayy66eef8c371.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.1 初始化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先需要初始化MML預測實例，一個示例對應一個模型，同時需要設置預測精度，模型路徑，Backend類型，以及Backend專用的一些配置信息。最後load方法將會使用指定的Backend加載模型。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"MMLMachineService *predictor = new mml_framework::MMLMachineService();MMLConfig config;config.precision = MMLConfig::FP32; \/\/ 預測精度config.modelUrl = modelDir; \/\/ 模型路徑config.machine_type = MMLConfig::MachineType::PaddleLite; \/\/ Backend類型MMLConfig::PaddleLiteConfig paddle_config; \/\/ Backend專用configconfig.machine_config.paddle_lite_config = paddle_config;predictor ->load(config);"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.2 前處理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前處理主要是對用戶的輸入數據進行操作，例如對圖片數據進行類型轉換，縮放等，MML提供了前後處理工具集，這些方法是經過ARM彙編或GPU優化的，如果是MML還未支持的操作，則可以使用OpenCV完成。對數據完成前處理後，可以通過predictor的getInputData方法創建MMLData數據容器，傳入的index參數表示模型的第幾個輸入，因爲有的模型需要多個輸入數據。然後通過resize設置輸入的維度信息，通過mutable_data申請所需內存，最後將用戶前處理後的數據拷貝到該內存即可。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"std::unique_ptr input = predictor->getInputData(0);input->mmlTensor->Resize({1, 3, 224, 224});auto *data = input->mmlTensor->mutable_data();"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.3 推理\/預測"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"設置好輸入之後，就可以進行預測了，非常簡單又令人激動的一步。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"predictor ->predict();"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.4 後處理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"獲取模型推理的結果，方式和前處理類似，通過predictor的getOutputData方法獲取MMLData類型的輸出，同樣可以傳入index表示獲取模型的第幾個輸出，然後通過shape獲取輸出的緯度信息，通過data獲取數據的指針，最後再通過MML的前後處理工具集進行相應操作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"std::unique_ptr output = predictor->getOutputData(0);auto shape = output ->mmlTensor->shape();auto *data = output ->mmlTensor->data();"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.5 釋放資源"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"delete predictor;"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"四、MML Kit"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MML 除了提供基礎的預測API以及工具集，還會提供多種通用AI能力SDK，包括超分辨率、手勢識別等，這些SDK組成了MML Kit。最初這些能力首先是在百度App部署上線，然而在遷移到其他App時，又有很多重複開發的工作，因此我們將這些通用AI能力進行封裝，使用方甚至不用關心任何模型相關的問題，不用關心預測Api的細節，只需集成相應的SDK即可，極大地降低了重複部署的成本。後續的文章中，我們會詳細介紹這部分的工作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"五、未來展望"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"未來，MML Core會持續完善和優化目前的功能和工具集，MML Kit也會提供更多的通用AI能力，並逐步在Github開源，爲業界提供易用、穩定、安全的移動端AI統一解決方案，全流程幫助移動端開發者部署AI能力。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule"},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"頭圖"},{"type":"text","text":"：Unsplash"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"作者"},{"type":"text","text":"：zhaojiaying"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"原文"},{"type":"text","text":"："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/7XUuL9_9DYoQXy6l_KKC8A","title":"","type":null},"content":[{"type":"text","text":"移動AI系列-百度App AI統一接入方案-MML"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"來源"},{"type":"text","text":"：百度App技術 - 微信公衆號 [ID：gh_59f5931152fe]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"轉載"},{"type":"text","text":"：著作權歸作者所有。商業轉載請聯繫作者獲得授權，非商業轉載請註明出處。"}]}]}