基於圖像識別的AI自動化測試框架：GAME AI SDK平臺解析

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一、GAME AI SDK概述"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自動化測試的好處顯而易見，但自動化測試的投入成本大卻是一個很大的障礙。爲了在遊戲測試中引入自動化測試, 騰訊互娛 TuringLab 實驗室研發了 Game AI SDK 自動化 AI 測試平臺。通過接入騰訊公司運營期或測試期的商業遊戲，不停的測試優化，最後推出這款 AI 自動化測試平臺。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，GAME AI SDK 已正式對外開源，"},{"type":"link","attrs":{"href":"https:\/\/github.com\/Tencent\/GameAISDK","title":"xxx","type":null},"content":[{"type":"text","text":"Git 開源地址"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同時 TuringLab 團隊出版"},{"type":"text","marks":[{"type":"strong"}],"text":"《AI自動化測試：技術原理、平臺搭建與工程實踐》"},{"type":"text","text":"一書，此書將 TuringLab 團隊基於深度學習的自動化AI測試框架基礎的開發與應用經驗，進行了詳細的描述，分享瞭如何將AI技術與自動化測試相結合完成測試需求的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於 GAME AI SDK 框架，可實現多種類型的手機遊戲 AI 開發（本文重點介紹手機遊戲方面）。目前我們已應用的遊戲類型有：射擊類(穿越火線，王牌戰士)， MMO(尋仙，龍之谷 ), 消除類(天天愛消除，消除者聯盟)，喫雞(和平精英)，賽車(QQ飛車)， MOBA(王者榮耀)， 格鬥(魂武者,DNF), 動作(魂鬥羅)，卡牌(聖鬥士星矢), 棋牌(天天德州)，跑酷(天天酷跑) ，體育(最強NBA)，飛行射擊(全民飛機大戰)等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同時也可以應用在手機 APP 及 Windows 應用程序方面。"},{"type":"text","marks":[{"type":"strong"}],"text":"不但適用於手機遊戲專項測試，手機 APP 專項測試，而且適用於 PC 端遊戲，軟件等專項測試"},{"type":"text","text":"（專項測試是指如手機兼容性測試，客戶端服務器端性能測試，版本回歸冒煙測試，跑圖覆蓋率及代碼覆蓋率測試，軟件健壯性及 crash 測試等這一類測試）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"二、GAME AI SDK的技術架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 是一個平臺工具系統，主要由四部分組成："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第一部分：AI SDK 平臺，系統的核心功能都集成在這個平臺裏面。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第二部分：tools 工具集，用戶可以根據系統提供的 SDKtool 進行 AI 相關配置操作，也可以根據需要，自行開發需要的工具，集成到 tools 工具集中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第三部分：環境摸擬（EM），主要提供手機遊戲的環境摸擬生成，用戶可以通過配置快速生成遊戲運行環境，這樣在訓練 DQN 等網絡時能節省很多時間，大幅提高訓練的效率。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第四部分：系統提供的 AI 模板庫，目前是按遊戲類型來劃分的，用戶可以根據自己的需求配置使用現有的 AI 模板庫，也可以在AI模板庫新增自己需要的模板。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"結構如下圖所示："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/f6\/f6f5ea158140920e47a1b4e30961017c.png","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1. GAME AI SDK平臺核心部分架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 平臺的核心部分的架構圖如圖所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/bb\/bbeb00dacedfad15f06ee820a48e619d.png","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 底層使用的是 TensorFlow 平臺和 OpenCV 開源庫，在此基礎上開發自動化系統、圖形圖像識別系統、AI 算法系統幾個核心功能模塊。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"自動化系統"},{"type":"text","text":"：主要負責數據接入部分，如果是截圖，則進行圖像採集；如果是通過API傳輸數據，則進行通信連接。同時，會負責將AI的動作輸出傳到手機上執行。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"圖形圖像UI模塊"},{"type":"text","text":"：負責遊戲 UI 的識別與處理。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"圖形圖像識別模塊"},{"type":"text","text":"：包括的平臺所有的圖形圖像識別算法，負責遊戲內所有的圖形圖像識別，並將識別結果傳到 AI 算法模塊。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"AI算法模塊"},{"type":"text","text":"：接收識別結果數據或者直接接收API輸入數據，根據 AI 網絡，輸出數個（0個到幾個）可能的動作，傳回到自動化系統。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 平臺通過自動化系統得到數據（如果是圖像，得到的是手機截屏；如果是接口 API，則直接得到 API 提供的數據）；把這部分數據輸入到圖形圖像模塊，圖形圖像模塊拿到數據後，通過配置好的識別算法，識別出結果，再把這個識別結果輸入到 AI 算法模塊，AI 算法根據自己的網絡，輸出數個（0個到幾個）可能的動作，傳回到手機上面去執行，這樣一次數據處理就完成了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"GAME AI SDK平臺流程"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"AI流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用戶的輸入數據有兩種方式："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"方式一：使用手機截圖，平臺通過圖像識別得到 AI 需要的數據，然後進行 AI 訓練。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"方式二：使用 API 接口得到 AI 需要的數據，然後進行 AI 訓練。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"方式一具有很好的通用性，不需要遊戲項目額外做任何的改動，缺點是圖像識別結果不一定完全準確。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"方式二需要遊戲項目額外提供數據 API 接口，因此需要投入額外的開發工作，但好處是數據是完全準確的，用戶可以根據項目需要進行適當的選擇。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整個 AI 流程如圖所示："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/1d\/1d5dd162fe2c3d35f91f5e0e3c392d6e.png","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step1：用戶通過數據預處理工具輸入數據（手機截圖或遊戲數據）；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step2：GAME AI SDK 讀取任務配置與AI算法配置；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step3：通過環境模擬平臺或在線直接進行 AI 訓練；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step4：將遊戲數據存儲到數據平臺中，方便下次訓練或者做數據分析使用（此步驟可選）。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"圖像識別任務流程圖"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 平臺的核心之一就是圖像識別任務。在配置圖像識別任務之前，如果此任務需要提前標註樣本（如需要使用 YOLO 算法），則需要進行數據預處理，可以使用平臺提供的 SDKTool 進行樣本標註，也可以使用如 LabeImage 這種軟件進行標註。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"樣本標註完成後，就可以使用平臺提供的 SDKTool 進行任務配置（詳細操作請參見下文第3章）。任務配置完成之後，就可以開始進行識別訓練（如果需要進行訓練的話）或直接進行識別了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖像識別任務流程圖如圖所示："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/9f\/9f415bd8fa88c8dad8a837ada65baa70.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step1：用戶使用標註工具進行樣本標註（識別算法需要標註樣本才進行此步驟）；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step2：在 SDKTool 中進行任務配置；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step3：開始任務預測試：在圖像識別模塊中進行圖像識別；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step4：如果用戶只需要做圖像識別，則直接將識別結果返回給用戶；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step5：如果用戶需要 AI 模塊，則將識別結果輸入到 AI 模塊；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step6：用戶如果覺得任務參數（包括圖像識別算法參數）需要調整，則重新調整任務參數，然後繼續進行任務預測試（step3），直到用戶認爲圖像識別效果達到要求爲止。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"GAME AI SDK平臺模塊結構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"圖像識別模塊"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖像識別模塊的輸入爲圖像幀數據，輸出爲圖像識別得到的結果數據。在圖像識別模塊內部，底層使用的是 TensorFlow 平臺和 Opencv 開源庫，在此基礎上，圖像識別模塊封裝了一些常用的識別算法（如 YOLO，模板匹配，像素檢測，特徵點匹配等），用戶可以直接通過配置使用這些算法，也可根據需要修改或者直接加入新的識別算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據遊戲本身的特點和需求，平臺在通用層封裝了一些遊戲中常用的識別算法，如遊戲中常見的數字、按鈕、血條等的識別，用戶可以直接使用，只需要在 SDKTool 工具裏面配置就可使用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖像識別本身的計算會根據圖像大小成倍增加，爲提高識別的性能，保證遊戲的響應時間，在圖像識別模塊中使用了多線程來提高圖像識別的性能，用戶可以根據機器的性能及對響應時間的要求，靈活配置多線程的數量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/96\/964641a442719e59fc6027d93d39dc3e.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖形圖像識別與跟蹤相關算法：此爲圖形圖像識別算法模塊，包括了一些常用圖形圖像識別算法，如像素檢測、特徵點匹配，梯度檢測等，也包括了一些基於機器學習的識別算法，如 YOLO，LeNet 等。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"comm：此模塊封裝了一些遊戲裏面需要經常用到的識別類型，數字類型：如遊戲中的分數、速度、距離、子彈數等；血條類型：如遊戲中角色的血量，怪物的血量，BOSS 的血量等。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"日誌管理：系統日誌管理模塊，輸出錯誤信息及程序運行信息，方便用戶查看程序運行狀態。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多線程管理：因爲圖形圖像識別的計算量會比較大，在複雜的識別任務或者要求高響應時間的應用場景下，可以使用更多的線程來進行圖像識別，多線程管理模塊可以很方便的設置多線程的運行數量。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖像識別邏輯處理：在此模塊中處理用戶配置的圖像識別任務。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"AI算法模塊"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AI 算法模塊的輸入有兩種方式：一種是圖像識別模塊識別的結果數據，一種是直接從 API 接口拿到的數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AI 算法模塊的底層也是使用 TensoFlow 平臺，與手機相關的操作使用 ADB 模塊，在此基礎上內置了 DQN 算法和 IM 算法。用戶通過配置 AI 算法模塊的配置文件及修改算法參數，可以方便地使用這兩個算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"DQN 算法使用方便，不需要標註數據，可以直接接入遊戲環境進行線上訓練，特點是需要訓練比較長的時間纔會有一定的效果，但泛化性相對 IM 來說要好些。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"IM 算法需要先記錄樣本數據（SDKTool 裏集成了樣本記錄功能，用戶可以直接使用），再根據樣本數據進行訓練，特點是訓練時間相對 DQN 來說會短很多，但泛化性沒有 DQN 好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"效果的好壞和記錄的樣本關係很大。用戶可根據項目需求來選擇使用哪種算法，或者自己開發新的算法。GAME AI SDK 提供了相關的 AI 算法接口，可以方便的擴展用戶自定義的 AI 算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"BeTree 是行爲樹，用戶可以根據項目業務需求，自行定義相關的行爲規則，控制 AI 的輸出。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"業務邏輯層主要是寫具體的遊戲業務相關的處理，比如用戶需要進行動作過濾，可以在此編寫相關的過濾規則，控制 AI 的輸出滿足遊戲業務測試的需求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/28\/28a87894c9d016f4326ff63f57e3fc2c.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"三、GAME AI SDK使用"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"環境安裝"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 環境安裝分本地環境安裝及鏡像方式安裝，如果需在進行本地二次開發，建議使用本地環境安裝方式，相關安裝操作說明請參見源碼文檔中環境部署一節的說明。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果只需要進行自動化測試相關測試操作，可以使用鏡像方式（此方法簡單容易上手，對測試人員比較友好）：從 GAME AI SDK 官網（"},{"type":"text","marks":[{"type":"italic"},{"type":"underline"}],"text":"http:\/\/aitest.qq.com"},{"type":"text","text":"）下載相應版本的鏡像安裝包及本地編譯包，本地解壓後根據說明文檔運行安裝腳本就可以進行環境安裝了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"SDKtool UI配置"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 不需要進行腳本編寫，只需進行少量的樣本採樣及標註就可以配置好。啓動 SDKTool 工具，連接好手機（或配置其他的圖像窗口），按相關要求進行配置，工具會自動生成相關的配置數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/2d\/2dd3a593cc9d3e188e2e298aeddb0772.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"SDKtool識別任務配置"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"識別任務是識別圖片中的某些關鍵數據信息，然後把這些數據信息做爲 AI 算法的輸入。配置操作與 UI 識別配置相同。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/e7\/e730020805a4ab6eaca0829dd4de2a0d.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"AI算法配置"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 平臺內置了 DQN 算法和 IM 算法，用戶可不需要編寫代碼或者腳本，通過簡單的參數配置，就可以使用平臺內置的這兩種算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其中 IM 算法訓練及採樣簡單快速（30 分鐘的樣本及 30 分鐘的離線訓練時間），適用一某一特定的遊戲場景，DQN 算法需要較長的在線衰弱時間，場景通用性較 IM 好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/f5\/f58b04bb8b60cf7ddb99b1469076a6ac.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"動作定義及配置"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據項目的具體動作，可以通過 SDKTool 定義相同的運作，這樣通過 AI 算法的訓練，AI agent 就可以根據圖片的輸入信息進行計算，最後輸出具體的運作，驅動遊戲像玩家玩遊戲一樣自動運行起來。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/e8\/e816f58b84b27ebe600a94ad82f16ca8.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"訓練及運行"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"完成工程所需要的相關配置後，就可以使用 SDKTool 工具進行 AI 算法的訓練及運行了。在 PC 機上連接好手機，就可以開始相關的訓練，訓練過程中的相關信息會通過界面顯示出來。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/fa\/face996392f58d2d1516a986a763b8e0.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"四、結語"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 的目標是開發一個比較通用的自動化平臺，讓用戶更專注於測試業務本身。它放棄了傳統的腳本自動化的方法，使用深度學習及圖像識別算法從圖像中提取中關鍵數據信息，做爲AI算法的輸入。通過大數據量的訓練得到一個通用的網絡模型，網絡模型本身就具有了較好的通用，可以在同一類項目中適用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"理論上，數據量越多，網絡模型本身的效果和通用性就越好。AI 算法及圖像識別算法發展越來越快、越來越好，GAME AI SDK 也在不斷努力優化與完善中，如果你有建議或需要技術支持，歡迎在評論區與我們聯繫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule"},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"頭圖：Unsplash"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者：周大軍"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文：https:\/\/mp.weixin.qq.com\/s\/bF5aEEIPcTGuECJ3K2-6QA"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文：基於圖像識別的AI自動化測試框架：GAME AI SDK平臺解析"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"來源：雲加社區 - 微信公衆號 [ID：QcloudCommunity]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"轉載：著作權歸作者所有。商業轉載請聯繫作者獲得授權，非商業轉載請註明出處。"}]}]}