三種分佈式人工智能主流框架方案對比

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在最近的幾年中,人工智能隨着神經網絡的突破,得到了巨大的發展,特別在圖像、分析、推薦等領域。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在人工智能快速發展的同時,計算規模不斷擴大、專家系統過於單一、神經網絡模型的靈活性、應用領域的複雜行等問題,也在不斷升級。在這樣的環境下,分佈式人工智能的發展被研究機構和大型企業提上日程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分佈式人工智能,可以解決集中化人工智能的幾個主要問題:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"規模化的計算問題。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"計算模型的拆分訓練。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多智能專家系統的協作。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多智能體博弈和訓練演化,解決數據集不足的問題。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"羣體智能決策和智能系統決策樹的組織,適應複雜的應用場景,比如工業、生物、航天等領域。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"適應物聯網和小型智能設備,聯合更多的計算設備和單元。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對現有的分佈式人工智能組件和框架的研究,可以看下幾個主流框架的方案。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"  "}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Tensorflow 的分佈式方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Tensorflow 底層數據是高性能的 gRPC 庫,分佈式集羣的組件主要是三個部分:client、master、worker process。運用它的組件,可以形成兩種主要的分佈式部署模式:"},{"type":"text","marks":[{"type":"strong"}],"text":"單機多卡(single process)"},{"type":"text","text":"和"},{"type":"text","marks":[{"type":"strong"}],"text":"多機多卡(multi-device process)"},{"type":"text","text":"。方案出於 Martin Abadi、Ashish Agarwal、Paul Barham 論文《TensorFlow:Large-Scale Machine Learning on Heterogeneous Distributed Systems》。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"它的部署和運作模式,如圖 1-1 所示:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/6e\/6eea4f9c75a156b01bce241236d33285.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖 1-1 Tensorflow 單機單卡與多級多卡部署模式"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於普通的機器學習,單機單卡就可以進行運算,在運算規模不斷升級的時候,就需要"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對 tensoflow 的訓練設計分佈式結構。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"機器學習在的參數訓練,主要是兩大過程:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用卷積參數訓練梯度"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據梯度再優化更新參數"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在大規模的計算過程中,就需要進行集羣計算,在小型規模的時候,可以使用單個機器,多個 CPU\/GPU 來進行計算,在規模越來越大的時候,可以使用多個機器進行並行計算。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 tensorflow 中分佈式集羣定義爲一個“集羣任務”(tf.train.ClusterSpec),將任務分佈在不同的“服務器”(tf.train.Server)中,服務器中包含了“master”運行進程和”worker“計算進程。在分佈式集羣運算中,主要有以下結構:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Task"},{"type":"text","text":":分佈式中的計算任務,每個 job 結構中,每個任務擁有唯一索引。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Job"},{"type":"text","text":":每個 Job 表示了一個完整的訓練目標,其中的 Task 都有唯一的目標,爲同一個訓練目標進行任務拆分。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Cluser"},{"type":"text","text":":集羣集合,其中管理了多個 Job,一般一個集羣對應了一個專門的神經網絡,其中不同的 job 負責了不同的目標,比如梯度計算、參數優化、任務訓練等。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Tensorflow Server"},{"type":"text","text":":處理服務節點過程,並且對外提供master server 和  worker server 的接口服務。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Master Server"},{"type":"text","text":":和遠程分佈式設備進行交互的服務,協調多個 worker server 工作。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Worker Service"},{"type":"text","text":":Tensorflow 圖計算過程中,任務進程處理的服務,單獨運行各自的任務。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Client"},{"type":"text","text":":客戶端,用來管理和啓動 AI 工作,與遠程服務集羣交互。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Tensorflow 的多機多卡分佈式模式,主要有以下幾種:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"In-graph 模式"},{"type":"text","text":":在多機多卡的模式下,各自的節點和運算服務,都通過單一的節點進行數據分發,擁有集中化的管理方式,最終數據歸併到統一的節點上。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Between-graph 模式"},{"type":"text","text":":訓練的數據保存在各自的節點上,不進行分發。每個節點的權重都是平等的,當計算節點各自計算完以後,需要告訴參數服務器,並且更新優化參數,是比較推薦的分佈式方式。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據 Tensorflow 的分佈式機器學習方案,我們可以總結它存在的一些缺點,如下所示:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Tensorflow 在分佈式的計算方式中,分爲了訓練和參數更新服務,極大得擴展了規模化的計算能力,但是在數據歸併和多級分層上,並沒有提供非常好的解決方案。沒有較好的計算模型,去提供多層次的分佈式結構。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Tensorflow 在分佈式的計算模型中,主要分爲了訓練任務、梯度計算、參數更新等,但是確沒有提供一個較好的計算模型構造方式,去提供分佈式 AI 所需要的任務分發、數據歸併、模型更新等能力。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Tensorflow 在分佈式的組織方式上,更像是一個平級層次的分佈式,最終統一歸併成爲一個最終模型,它最終完成的是一個集中化的單一專家系統,無法提供羣體智能能力,也無法形成多羣體博弈對抗的協調決策,無法支持多專家系統的兼容和協調。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Tensorflow 需要運行在高性能計算機,高性能的 GPU 下,很難支持小型設備、物聯網等多設備的邊緣化集羣計算。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Spark 分佈式機器學習"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面提到,Tensorflow 在機器學習並行計算的多層結構上,沒有較好的解決方案,Spark 根據它在大數據領域的分佈式框架,很好的解決了分層任務劃分的處理。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 Spark 中,計算模型可以設計成爲有向無環圖 DAG,無環圖的頂點是 RDD,它是 Spark的核心組件。RDD 是一種彈性的分佈式數據集,它可以支持多個 RDD 分片的依賴、變換(transformation)和動作 (action),它可以從從 RDD A 變換成爲 RDD B,這個 transformation 就是 DAG 無環圖的邊。通過 DAG 的表示方法,表示計算模型,並且編譯成爲 Stage。這個模型的表示,如圖 1-2 所示:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/15\/154baf0780bb0d456271250e8b4a2631.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖 1-2 Spark 計算模型和 Stage 形成的 DAG 圖"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Spark 運用它的分佈式機制,可以支持多個 Stage 並行計算,也能支持 Stage 下屬 Stage 模型分層結構。Spark 基於它的 master-worker 架構,可以將 DAG 中的 stage 分割,指定到不同的機器上面執行任務。它的驅動器 driver 負責協調任務和調度器組件 scheduler,調度器分爲 DAG 和 Task 調度器,用來分配 Task 到不同的運算單元。它的分佈式並行運算流程,如圖 1-3 所示:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/0c\/0c014f3ac2b76983836863a22c611556.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖 1-3 Spark 分佈式並行計算分層"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上圖表示了在分佈式 AI 的訓練環節,通過 Driver 規劃整體模型,通過節點 Executor 端分別計算,再回歸彙總給 Driver,它們的具體要點如下所示:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個具體的 AI 系統總結爲一個工作流 Work Process,其中包含了許多 action 算子操作。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"action 算子將不同的工作類別劃分爲不同 Job。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Job 根據模型的寬以來,劃分給不同的 Stage 小模型。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Stage 的內部也可以再劃分成爲多個小型的 Stage 模型,如圖 1-2 所示。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Stage 將功能相同的,DAG 中的邊運算歸納爲不同的 Task。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Task 可以提交 Executor 做最小運算單元的運算,最終反饋給 Driver 進行模型更新和存儲。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過 Spark 的分佈式計算框架與機器學習 caffe 運算庫進行結合,伯克利大學 Michael I. Jordan 組發佈了論文《SparkNet: Training Deep Network in Spark》,他們開發了 SparkNet 庫,它是基於 Spark 的深度神經網絡架構,主要能力如下所示:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"提供易用的,用於神經網絡的接口,可以快速訪問 RDD。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"提供 Scala 的接口,可以與 caffe 進行交互。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"提供輕量級的卷積 tensor 庫。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"提供簡單的並行機制和通訊機制。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即插即拔,易於部署。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"兼容現有的 caffe 模型。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據 Spark 的分佈式機器學習機制,我們可以總結它存在的一些缺點,如下所示:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Spark 支持大規模的模型計算分層,但是計算消耗較大,不適合用在較小規模的分佈式方案。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Spark 在節點和節點之間的數據路由不是非常靈活。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Spark 難以在小型化設備進行聯合學習運算,無法部署在小型化設備之上。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Spark 的框架是一個並行計算框架,它的模式是任務確認調度-模型劃分-分片計算-歸併總結,更適合大數據集的機器學習,不適合多節點博弈等強化學習方法。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Google 聯合學習方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對小型化設備,google 在一篇文章《Federated Learning: Collaborative Machine"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Learning without Centralized Training Data》中,提出了聯邦聯合學習的概念。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"它的工作原理如下所示:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"手機或者其他小型設備,下載雲端的共享模型。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"每個小型設備的用戶,通過自己的歷史數據來訓練和更新模型。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"將用戶個性化更新後的模型,抽取成爲一個小的更新文件。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"提取模型的差異化部分,進行加密,上傳到雲端。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","text":"在雲端將新用戶的差異化模型和其他用戶模型,進行 Average 平均化,然後更新改善現有共享模型。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這樣工作的好處有如下部分:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"聚合了邊緣的小型化設備(比如手機),增加了 AI 的數據來源和計算能力來源。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在機器學習的模型結果上,更加適應廣泛用戶的行爲數據模型。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不同羣體之間,可以產生博弈和不斷強化的模型結果,模型可以在廣泛的分佈式基礎下,不斷迭代更新。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文轉載自:金科優源匯(ID:jkyyh2020)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接:"},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/t7zS7RwjW4j3m2SOllrL1A","title":"xxx","type":null},"content":[{"type":"text","text":"三種分佈式人工智能主流框架方案對比"}]}]}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章