Flutter UI自動化測試技術方案選型與探索

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Flutter頁面無法直接使用Native測試工具定位元素，給自動化測試帶來很多不便。雖然Google官方推出了Flutter driver 和 Integration test，但是在實際使用中存在以下問題："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不適用於混合棧APP，雖然appium中有相關的driver，但是無法切換環境。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"元素定位能力相對薄弱。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"依賴於VMService，需要構建Profile或Debug包。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於以上因素，我們並沒有直接使用Google官方推出的工具，而是選擇基於Native測試工具去擴展Flutter頁面的測試能力。本文對Flutter driver 和Integration test的原理和實現進行了分析，同時簡單介紹閒魚在UI自動化測試的嘗試方案。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Flutter driver"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最早接觸flutter自動化測試時，先嚐試使用appium框架去驅動APP，當我們使用inspect功能去dump頁面元素時發現很多元素會被合併成一個區域塊，然後點擊的時候只能通過xpath定位，想定位到某些具體的元素會比較困難，並且xpath其實是容易改變的，代碼可維護性能力差。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因爲上述原因，我們開始調研Flutter官方提供的測試工具——flutter driver。一開始使用該框架的時候發現它只能適用於純Flutter應用，對於混合棧應用並不適應，但是它底層提供的元素定位能力或許對我們有用，於是我們對它的源碼進行了剖析，該框架的原理圖1如下所示。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/e6\/e6cdcd682677fb1c90f469e3f8536baa.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖1 flutter driver原理圖 "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整個框架的流程交互比較簡單，測試腳本在運行時，首先利用FlutterDriver.connect()來連接VMService獲取相關的isolate，之後通過websocket來傳輸操作過程以及數據獲取。其中測試腳本側的所有操作都是被序列化爲json字符串通過websocket傳遞給ioslate來轉換爲命令在APP側執行，例如我們想要獲取某個組件的文本內容，其最終生成的json結構體如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"{\n \"jsonrpc\":\"2.0\",\n \"id\":5,\n \"method\":\"ext.flutter.driver\",\n \"params\":{\n \"finderType\":\"ByValueKey\",\n \"keyValueString\":\"counter\",\n \"keyValueType\":\"String\",\n \"command\":\"get_text\",\n \"isolateId\":\"isolates\/4374098363448227\"\n }\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"瞭解上述原理後，就可以通過構造協議格式，在任何語言、測試框架下都能夠去驅動flutter測試，所以我們對這個協議進行了封裝，使用Python進行驅動，這樣可以在使用uiautomator2和facebook-wda的基礎上來測試flutter頁面，以滿足flutter混合棧應用的測試需求。最終的實現代碼demo如下。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"from flutter_driver.finder import FlutterFinder\nfrom flutter_driver.flutter_driver import FlutterDriver\nimport uiautomator2 as u2\n\nif __name__ == \"__main__\":\n d = u2.connect()\n driver = FlutterDriver(d)\n if pageFlutter is True: # 如果是flutter，則使用flutter driver進行驅動\n driver.connect(\"com.it592.flutter_app\")\n finder = FlutterFinder.by_value_key(\"input\")\n driver.tap(finder)\n time.sleep(1)\n print(driver.getText(FlutterFinder.by_value_key(\"counter\")))\n else:\n d(text=\"increase\").click()\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們嘗試使用該套框架，發現其實flutter driver底層提供的能力相對比較薄弱，並不能完全滿足我們的需求，主要問題如下："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不能批量操作元素，一旦finder定位到的元素超過1個時，就會拋出異常。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"很多時候開發同學不寫key，元素定位也沒那麼方便。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因爲flutter沒有inspect工具dump元素，所以只能利用結合源碼去寫腳本，代碼維護成本比較高。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"官方已經放棄維護該項目，所以後續估計也不會有新功能支持。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"integration_test"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面提到，flutter官方放棄維護Flutter driver，並推出新的測試框架integration_test，那麼這個框架會不會對混合棧應用予以支持呢，事實上試用了之後發現事情並沒有我們想的那麼美妙。在官方文檔裏有這麼一句話“該軟件包可在設備和模擬器上對Flutter代碼進行自驅動測試”。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"integration_test底層的元素操作和定位還是基於flutter_test去驅動的，其優勢主要如下："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"測試腳本可以使用各種Flutter的API。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"打包ipa、apk後就能在 Firebase Test Lab等設備羣上運行測試，不需要額外驅動。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"integration_test的每個頁面之間測試無關聯，可以實現單個頁面級別的測試。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是由於底層元素定位和Flutter driver的是一致的，所以Flutter driver存在的問題依舊存在，同時還存在其他侷限問題："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"測試腳本打包到APP中，每次修改腳本都需要重新打包。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對端到端測試不夠友好，需要額外函數來等待數據加載完畢。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不適合全鏈路級別的頁面測試。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可擴展性弱"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於以上問題，不滿足我們的使用需求，所以我們只是做了簡單預研，並沒有深入瞭解和應用。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"閒魚UI自動化測試方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"學習Flutter官方推出的相關測試框架之後，我們開始思考閒魚UI自動化到底要怎麼走？是站在官方的肩膀上去造輪子還是複用現有的原生自動化測試能力去擴展Flutter測試能力。在綜合考慮投入成本以及測試腳本的維護難度後，我們選擇使用圖像處理技術來擴充原生自動化框架對Flutter頁面的測試能力支持，整個測試方案架構如圖2所示。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/00\/0033d2edd679f473f5237b31d52245f7.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖2 閒魚UI自動化測試方案架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Flutter的元素不是完全不能被uiautomator2和facebook-wda識別，所以編寫測試腳本時只需要處理不能被識別的元素即可。對於有name、label以及xpath不易改變的元素定位，我們優先使用原生定位能力進行定位操作，其他元素則直接使用圖像處理技術進行定位操作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在處理無法使用原生能力定位的元素時，我們優先使用ocr文字匹配來進行定位，準確率較高，不容易受分辨率的影響，對於純圖片則通過圖片查找的方式進行定位。對於一些常見的元素控件例如商品卡片、價格、icon、頭像等，我們構建一個訓練集，使用圖像分類來判斷元素的類型，從而實現常用控件的定位。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"UI自動化面臨最大的問題就是——隨着版本的迭代，測試腳本也需要進行不斷迭代。所以在方案選型和腳本編寫過程中需要考慮到腳本的健壯性以及可維護性。我們在實際腳本開發中將頁面元素封裝到單獨的類中，並與測試邏輯分離，從而保證後期元素迭代時只需要修改對應的頁面元素即可，減少維護成本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/4c\/4caadf1a8ace283397343c479a01b8c1.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖3 腳本分層結構 "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"閒魚性能自動化測試的相關UI操作已經使用該方案，在腳本編寫時，並不需要區分當前頁面是什麼類型。我們的腳本已經穩定運行500+次，成功率超過98%。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/5e\/5e4f30e1879296158f124afc1df76604.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖4 方案對比 "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從圖4可以看出，無論是flutter driver還是integration test對混合棧的支持不夠成熟，但是flutter driver可以進行一些擴展，對於純Flutter應用而言，採用該方案能夠基本滿足測試需求，而integration test相對沒有那麼成熟，對於混合棧應用的測試，可能還是需要考慮混合棧的場景切換成本，使用一些ocr技術去做一些擴充可能成本更低，收益更大。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文轉載自：閒魚技術（ID：XYtech_Alibaba）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/v-ZfDglV4GaOqu04AO_3yA","title":"xxx","type":null},"content":[{"type":"text","text":"Flutter UI自動化測試技術方案選型與探索"}]}]}]}