基于图像识别的AI自动化测试框架：GAME AI SDK平台解析

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一、GAME AI SDK概述"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自动化测试的好处显而易见，但自动化测试的投入成本大却是一个很大的障碍。为了在游戏测试中引入自动化测试, 腾讯互娱 TuringLab 实验室研发了 Game AI SDK 自动化 AI 测试平台。通过接入腾讯公司运营期或测试期的商业游戏，不停的测试优化，最后推出这款 AI 自动化测试平台。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，GAME AI SDK 已正式对外开源，"},{"type":"link","attrs":{"href":"https:\/\/github.com\/Tencent\/GameAISDK","title":"xxx","type":null},"content":[{"type":"text","text":"Git 开源地址"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同时 TuringLab 团队出版"},{"type":"text","marks":[{"type":"strong"}],"text":"《AI自动化测试：技术原理、平台搭建与工程实践》"},{"type":"text","text":"一书，此书将 TuringLab 团队基于深度学习的自动化AI测试框架基础的开发与应用经验，进行了详细的描述，分享了如何将AI技术与自动化测试相结合完成测试需求的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基于 GAME AI SDK 框架，可实现多种类型的手机游戏 AI 开发（本文重点介绍手机游戏方面）。目前我们已应用的游戏类型有：射击类(穿越火线，王牌战士)， MMO(寻仙，龙之谷 ), 消除类(天天爱消除，消除者联盟)，吃鸡(和平精英)，赛车(QQ飞车)， MOBA(王者荣耀)， 格斗(魂武者,DNF), 动作(魂斗罗)，卡牌(圣斗士星矢), 棋牌(天天德州)，跑酷(天天酷跑) ，体育(最强NBA)，飞行射击(全民飞机大战)等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同时也可以应用在手机 APP 及 Windows 应用程序方面。"},{"type":"text","marks":[{"type":"strong"}],"text":"不但适用于手机游戏专项测试，手机 APP 专项测试，而且适用于 PC 端游戏，软件等专项测试"},{"type":"text","text":"（专项测试是指如手机兼容性测试，客户端服务器端性能测试，版本回归冒烟测试，跑图覆盖率及代码覆盖率测试，软件健壮性及 crash 测试等这一类测试）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"二、GAME AI SDK的技术架构"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 是一个平台工具系统，主要由四部分组成："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第一部分：AI SDK 平台，系统的核心功能都集成在这个平台里面。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第二部分：tools 工具集，用户可以根据系统提供的 SDKtool 进行 AI 相关配置操作，也可以根据需要，自行开发需要的工具，集成到 tools 工具集中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第三部分：环境摸拟（EM），主要提供手机游戏的环境摸拟生成，用户可以通过配置快速生成游戏运行环境，这样在训练 DQN 等网络时能节省很多时间，大幅提高训练的效率。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第四部分：系统提供的 AI 模板库，目前是按游戏类型来划分的，用户可以根据自己的需求配置使用现有的 AI 模板库，也可以在AI模板库新增自己需要的模板。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"结构如下图所示："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/f6\/f6f5ea158140920e47a1b4e30961017c.png","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1. GAME AI SDK平台核心部分架构"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 平台的核心部分的架构图如图所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/bb\/bbeb00dacedfad15f06ee820a48e619d.png","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 底层使用的是 TensorFlow 平台和 OpenCV 开源库，在此基础上开发自动化系统、图形图像识别系统、AI 算法系统几个核心功能模块。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"自动化系统"},{"type":"text","text":"：主要负责数据接入部分，如果是截图，则进行图像采集；如果是通过API传输数据，则进行通信连接。同时，会负责将AI的动作输出传到手机上执行。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"图形图像UI模块"},{"type":"text","text":"：负责游戏 UI 的识别与处理。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"图形图像识别模块"},{"type":"text","text":"：包括的平台所有的图形图像识别算法，负责游戏内所有的图形图像识别，并将识别结果传到 AI 算法模块。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"AI算法模块"},{"type":"text","text":"：接收识别结果数据或者直接接收API输入数据，根据 AI 网络，输出数个（0个到几个）可能的动作，传回到自动化系统。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 平台通过自动化系统得到数据（如果是图像，得到的是手机截屏；如果是接口 API，则直接得到 API 提供的数据）；把这部分数据输入到图形图像模块，图形图像模块拿到数据后，通过配置好的识别算法，识别出结果，再把这个识别结果输入到 AI 算法模块，AI 算法根据自己的网络，输出数个（0个到几个）可能的动作，传回到手机上面去执行，这样一次数据处理就完成了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"GAME AI SDK平台流程"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"AI流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用户的输入数据有两种方式："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"方式一：使用手机截图，平台通过图像识别得到 AI 需要的数据，然后进行 AI 训练。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"方式二：使用 API 接口得到 AI 需要的数据，然后进行 AI 训练。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"方式一具有很好的通用性，不需要游戏项目额外做任何的改动，缺点是图像识别结果不一定完全准确。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"方式二需要游戏项目额外提供数据 API 接口，因此需要投入额外的开发工作，但好处是数据是完全准确的，用户可以根据项目需要进行适当的选择。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整个 AI 流程如图所示："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/1d\/1d5dd162fe2c3d35f91f5e0e3c392d6e.png","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step1：用户通过数据预处理工具输入数据（手机截图或游戏数据）；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step2：GAME AI SDK 读取任务配置与AI算法配置；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step3：通过环境模拟平台或在线直接进行 AI 训练；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step4：将游戏数据存储到数据平台中，方便下次训练或者做数据分析使用（此步骤可选）。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"图像识别任务流程图"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 平台的核心之一就是图像识别任务。在配置图像识别任务之前，如果此任务需要提前标注样本（如需要使用 YOLO 算法），则需要进行数据预处理，可以使用平台提供的 SDKTool 进行样本标注，也可以使用如 LabeImage 这种软件进行标注。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"样本标注完成后，就可以使用平台提供的 SDKTool 进行任务配置（详细操作请参见下文第3章）。任务配置完成之后，就可以开始进行识别训练（如果需要进行训练的话）或直接进行识别了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"图像识别任务流程图如图所示："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/9f\/9f415bd8fa88c8dad8a837ada65baa70.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step1：用户使用标注工具进行样本标注（识别算法需要标注样本才进行此步骤）；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step2：在 SDKTool 中进行任务配置；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step3：开始任务预测试：在图像识别模块中进行图像识别；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step4：如果用户只需要做图像识别，则直接将识别结果返回给用户；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step5：如果用户需要 AI 模块，则将识别结果输入到 AI 模块；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Step6：用户如果觉得任务参数（包括图像识别算法参数）需要调整，则重新调整任务参数，然后继续进行任务预测试（step3），直到用户认为图像识别效果达到要求为止。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"GAME AI SDK平台模块结构"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"图像识别模块"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"图像识别模块的输入为图像帧数据，输出为图像识别得到的结果数据。在图像识别模块内部，底层使用的是 TensorFlow 平台和 Opencv 开源库，在此基础上，图像识别模块封装了一些常用的识别算法（如 YOLO，模板匹配，像素检测，特征点匹配等），用户可以直接通过配置使用这些算法，也可根据需要修改或者直接加入新的识别算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根据游戏本身的特点和需求，平台在通用层封装了一些游戏中常用的识别算法，如游戏中常见的数字、按钮、血条等的识别，用户可以直接使用，只需要在 SDKTool 工具里面配置就可使用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"图像识别本身的计算会根据图像大小成倍增加，为提高识别的性能，保证游戏的响应时间，在图像识别模块中使用了多线程来提高图像识别的性能，用户可以根据机器的性能及对响应时间的要求，灵活配置多线程的数量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/96\/964641a442719e59fc6027d93d39dc3e.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"图形图像识别与跟踪相关算法：此为图形图像识别算法模块，包括了一些常用图形图像识别算法，如像素检测、特征点匹配，梯度检测等，也包括了一些基于机器学习的识别算法，如 YOLO，LeNet 等。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"comm：此模块封装了一些游戏里面需要经常用到的识别类型，数字类型：如游戏中的分数、速度、距离、子弹数等；血条类型：如游戏中角色的血量，怪物的血量，BOSS 的血量等。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"日志管理：系统日志管理模块，输出错误信息及程序运行信息，方便用户查看程序运行状态。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多线程管理：因为图形图像识别的计算量会比较大，在复杂的识别任务或者要求高响应时间的应用场景下，可以使用更多的线程来进行图像识别，多线程管理模块可以很方便的设置多线程的运行数量。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"图像识别逻辑处理：在此模块中处理用户配置的图像识别任务。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"AI算法模块"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AI 算法模块的输入有两种方式：一种是图像识别模块识别的结果数据，一种是直接从 API 接口拿到的数据。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AI 算法模块的底层也是使用 TensoFlow 平台，与手机相关的操作使用 ADB 模块，在此基础上内置了 DQN 算法和 IM 算法。用户通过配置 AI 算法模块的配置文件及修改算法参数，可以方便地使用这两个算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"DQN 算法使用方便，不需要标注数据，可以直接接入游戏环境进行线上训练，特点是需要训练比较长的时间才会有一定的效果，但泛化性相对 IM 来说要好些。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"IM 算法需要先记录样本数据（SDKTool 里集成了样本记录功能，用户可以直接使用），再根据样本数据进行训练，特点是训练时间相对 DQN 来说会短很多，但泛化性没有 DQN 好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"效果的好坏和记录的样本关系很大。用户可根据项目需求来选择使用哪种算法，或者自己开发新的算法。GAME AI SDK 提供了相关的 AI 算法接口，可以方便的扩展用户自定义的 AI 算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"BeTree 是行为树，用户可以根据项目业务需求，自行定义相关的行为规则，控制 AI 的输出。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"业务逻辑层主要是写具体的游戏业务相关的处理，比如用户需要进行动作过滤，可以在此编写相关的过滤规则，控制 AI 的输出满足游戏业务测试的需求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/28\/28a87894c9d016f4326ff63f57e3fc2c.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"三、GAME AI SDK使用"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"环境安装"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 环境安装分本地环境安装及镜像方式安装，如果需在进行本地二次开发，建议使用本地环境安装方式，相关安装操作说明请参见源码文档中环境部署一节的说明。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果只需要进行自动化测试相关测试操作，可以使用镜像方式（此方法简单容易上手，对测试人员比较友好）：从 GAME AI SDK 官网（"},{"type":"text","marks":[{"type":"italic"},{"type":"underline"}],"text":"http:\/\/aitest.qq.com"},{"type":"text","text":"）下载相应版本的镜像安装包及本地编译包，本地解压后根据说明文档运行安装脚本就可以进行环境安装了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"SDKtool UI配置"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 不需要进行脚本编写，只需进行少量的样本采样及标注就可以配置好。启动 SDKTool 工具，连接好手机（或配置其他的图像窗口），按相关要求进行配置，工具会自动生成相关的配置数据。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/2d\/2dd3a593cc9d3e188e2e298aeddb0772.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"SDKtool识别任务配置"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"识别任务是识别图片中的某些关键数据信息，然后把这些数据信息做为 AI 算法的输入。配置操作与 UI 识别配置相同。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/e7\/e730020805a4ab6eaca0829dd4de2a0d.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"AI算法配置"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 平台内置了 DQN 算法和 IM 算法，用户可不需要编写代码或者脚本，通过简单的参数配置，就可以使用平台内置的这两种算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其中 IM 算法训练及采样简单快速（30 分钟的样本及 30 分钟的离线训练时间），适用一某一特定的游戏场景，DQN 算法需要较长的在线衰弱时间，场景通用性较 IM 好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/f5\/f58b04bb8b60cf7ddb99b1469076a6ac.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"动作定义及配置"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根据项目的具体动作，可以通过 SDKTool 定义相同的运作，这样通过 AI 算法的训练，AI agent 就可以根据图片的输入信息进行计算，最后输出具体的运作，驱动游戏像玩家玩游戏一样自动运行起来。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/e8\/e816f58b84b27ebe600a94ad82f16ca8.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"训练及运行"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"完成工程所需要的相关配置后，就可以使用 SDKTool 工具进行 AI 算法的训练及运行了。在 PC 机上连接好手机，就可以开始相关的训练，训练过程中的相关信息会通过界面显示出来。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/fa\/face996392f58d2d1516a986a763b8e0.webp","alt":"Image","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"四、结语"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GAME AI SDK 的目标是开发一个比较通用的自动化平台，让用户更专注于测试业务本身。它放弃了传统的脚本自动化的方法，使用深度学习及图像识别算法从图像中提取中关键数据信息，做为AI算法的输入。通过大数据量的训练得到一个通用的网络模型，网络模型本身就具有了较好的通用，可以在同一类项目中适用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"理论上，数据量越多，网络模型本身的效果和通用性就越好。AI 算法及图像识别算法发展越来越快、越来越好，GAME AI SDK 也在不断努力优化与完善中，如果你有建议或需要技术支持，欢迎在评论区与我们联系。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule"},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"头图：Unsplash"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者：周大军"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文：https:\/\/mp.weixin.qq.com\/s\/bF5aEEIPcTGuECJ3K2-6QA"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文：基于图像识别的AI自动化测试框架：GAME AI SDK平台解析"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"来源：云加社区 - 微信公众号 [ID：QcloudCommunity]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"转载：著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。"}]}]}