愛奇藝iOS穩定性測試實踐

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"穩定性測試是長時間持續運行APP，以驗證應用是否穩定的測試。它可以有效發現APP長時間運行下的偶發閃退、內存泄露、性能變差等問題。iOS端通常由蘋果系統的API快速執行點擊事件，開展穩定性測試，類似的優秀工具如FastMonkey等雖然有諸多好處，但是作爲長期運行的測試服務系統，還需要調整功能以適應企業級測試場景，無法解決通過外部請求定製事件執行序列、無法動態設置啓動參數、截圖存在本地將導致磁盤佔用過大等問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"愛奇藝測試團隊在iOS穩定性測試方面展開了不懈的探索，也積累了一些相關經驗，希望借本文跟大家分享在【iOS穩定性測試】實踐和優化過程中的心得和體會，也藉此機會拋磚引玉，徵求同行的更多探討。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"一、方案實踐","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.1基礎框架","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"愛奇藝iOS穩定性測試基於現有的雲真機體系，總體分爲三大塊。向下是設備管理，向上是產出物彙總，核心是測試策略。基礎框架如下圖1，真機設備通過驅動層接入遠程控制體系，統一由後端調度管理，核心策略通過設備驅動與真機設備交互，模擬用戶行爲開展測試，測試過程中產生的數據統一彙總到後端，由後端生成測試報告並反饋給用戶。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ff/ffe28bdd8aa5d19c2967de770d567d51.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖1. 系統框架","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"系統結構如上，下面重點討論策略選擇的問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"常用的穩定性測試方案，有Monkey測試、錄製回放或者按元素遍歷的方式等。其中Monkey測試實現成本較低，但有的頁面元素較少，盲目的操作經常點不到元素，導致測試效率較低；對於長期執行錄製回放的方案，其執行路徑固定且與業務功能耦合，可能需要長期維護業務邏輯，不利於各業務線快速拓展穩定性測試場景；而按元素遍歷操作的方案，可以較好的處理上述方案的問題，但是實現成本比Monkey策略高，需要測試團隊持續投入研發力量。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們嘗試了隨機滑動點擊策略、按元素點擊的遍歷策略，下面分析下兩種方案的實現細節和難點。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.2生成測試事件","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們抽象出一個模塊叫事件生成器，主要作用是持續產生用戶事件流作用於APP，以達到驗證APP穩定性的目的。本文重點介紹隨機產生和根據頁面元素產生兩種模式。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.2.1隨機產生事件","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨機事件方案的典型代表是Monkey，它向系統發送隨機的用戶事件流（如點擊、輸入、滑動等），實現對正在開發的應用程序進行穩定性測試。該方案通過XCTest提供的接口就可以快速模擬相關操作，實現成本相對較低。早期爲了快速打通測試流程，驗證方案有效性，我們實現了Monkey測試的方案。如圖2所示：先獲取本次任務期望各事件發生的佔比配置（涉及重啓APP、按home鍵、切換橫豎屏、點擊事件、滑動事件、後退等事件），再按概率指定本次執行哪種事件。組裝好需要執行的事件，最後請求具體的驅動服務以執行事件。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b9/b962723297f9be94799a93176e2825a6.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖2 隨機事件生成器","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.2.2根據頁面元素產生事件","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨機策略雖然點擊頻率高，但是無效事件佔比高達90%。分析測試報告發現當頁面可點擊元素較少時，隨機生成器產生的事件就會觸發大量無效操作。如果可以根據頁面的可操作元素進行遍歷操作，就可以大大減少無效事件的產生。因此嘗試使用按元素點擊的策略，執行深度/廣度優先遍歷策略，提升測試效率。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那iOS端如何獲取元素、記錄執行路徑以及按元素來遍歷的方式執行測試策略呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"識別頁面上的元素","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"按元素點擊首先要解決元素識別的問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"方法1，","attrs":{}},{"type":"text","text":"被測APP集成SDK，接入成本高，且正式包一般不允許帶入；","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"方法2，","attrs":{}},{"type":"text","text":"從DOM樹解析元素，存在DOM樹元素累積的情況，當頁面分頁時，無法區分僅當前頁的元素，影響元素解析準確性。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這兩種方法都存在一定的問題，因此我們把眼光轉向了","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"方法3","attrs":{}},{"type":"text","text":"，AI圖像識別元素。AI圖像識別作爲一項相對成熟的技術可以有效避免上述問題，並提供相對準確的元素數據。因此在元素生成方面，我們藉助愛奇藝內部提供的AI服務，在事件操作前將畫面截屏傳遞給AI服務，快速識別絕大多數元素區域，如下圖3所示。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/37/37d1b4a207abd4b292ed5f07fda44d24.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖3 AI服務識別頁面的元素","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"對多個頁面去重，定位當前所處頁面","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"解決了元素識別的問題後，下一步就是解決元素遍歷的邏輯。爲提高遍歷效率，遍歷邏輯需要儘可能的避免多次進到重複的頁面遍歷。爲了定位到當前所處頁面以記錄執行路徑，順帶避免相同頁面重複請求AI服務，需要對當前頁面是否曾經進入過進行判斷，我們考慮的方法有2個：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"方法1，","attrs":{}},{"type":"text","text":"從DOM樹提取關鍵信息來生成指紋，以指紋定位當前所處的頁面。但經過實踐發現獲取DOM樹的接口耗時不穩定，經常耗時>4s，不利於快速定位頁面。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"方法2","attrs":{}},{"type":"text","text":"，利用事件操作前後的屏幕截圖生成像素指紋，來定位當前所處的頁面。實踐發現通過此方法定位頁面，速度可達到毫秒級別。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"大致流程如圖4，先獲取屏幕截圖，再把截圖處理灰度化、縮放爲8*9的縮略圖，最後再生成包含圖像信息的指紋。通過計算指紋之間相似度是否超過閾值來判定圖片是否爲同一張，從而判斷處在哪一頁。優點是後續可直接通過指紋計算頁面之間的相似度，速度較快，也通過提高閾值避免局部細微變化干擾判斷結果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/19/19a9aff1cea7507e19373a572f991711.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖4 以屏幕截圖生成指紋判斷相似度","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"返回上一個頁面","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"廣度優先算法每個步驟點擊成功都需要返回上頁，繼續點擊該頁其他元素；深度策略點到沒元素點擊的頁面也需要嘗試返回以繼續點擊，所以實現通用返回策略是有必要的。不同於Android有物理按鍵強制返回，iOS端需要自己來實現一套返回方法。結合被測APP特性，我們封裝的返回邏輯是，大部分頁面可左滑返回上一頁，禁用左滑的頁面則靠以下方式來處理。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"比如H5頁面的關閉導航按鈕等，可識別DOM樹中屬性包含back/close字樣的元素爲返回、關閉按鈕，嘗試點擊以返回上一頁；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"部分頁面導航按鈕以“返回”文字呈現，方法1就找不到這種退出按鈕。我們把屏幕截圖上傳給OCR文字識別服務，識別屏幕中的“返回”文字。點擊識別到的返回文字以嘗試返回上一頁；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"部分廣告浮層的關閉按鈕是“X”圖樣的關閉圖片，上述兩方法都不能處理。我們則批量蒐集此類關閉圖樣訓練了可以識別關閉按鈕位置的AI工具，通過點擊識別的元素位置以返回上一頁。","attrs":{}}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"所以我們如示意圖5，按元素深度優先遍歷策略單步事件執行。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/4a/4ac655a17e6ecff758651f19c2c77c59.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖5 元素事件生成器","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.3處理運行時的突發干擾","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實踐發現，自動化測試運行時可能存在偶發的彈窗阻擋點擊目標元素、可能誤操作使APP退到了後臺，也可能在某頁卡住無法退出。如何處理各種阻礙測試的異常，確保APP測試過程中可以觸達更多的頁面，提升測試覆蓋程度呢 ？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.3.1部分應用進不去首頁的解決","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有的APP必需先執行登錄等操作才能進入首頁開始測試任務。各個業務線情況多種多樣，所以框架支持定製化腳本的運行。允許讓用戶定製差異化的前置場景，在運行穩定性測試前先執行定製腳本，再開始穩定性測試。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.3.2偶然的彈窗面板處理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"APP運行過程中出現的彈窗多種多樣，大致可分爲如下表格6中幾類：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d0/d0eaf45a2fb6ef6d4fc5397218794fd4.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"表格6 彈窗情況分類","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"系統彈窗檢測和處理相對簡單，只需要提前設置要識別的文本，WDA接口通過文本屬性判斷元素存在則直接點擊掉。但是當業務線越來越多，需要設置的文本數量也急劇增長，每步事件操作前都需要循環判斷是否有滿足設定文本的彈窗，時間損耗太大。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"研究發現iOS的“正則表達式”-NSPredicate方式，支持一次請求判斷多個元素是否存在，同時正則表達式的方式可以根據需求快速擴展，有效的避免文本的窮盡枚舉。基於這樣的新的思路，每步檢測彈窗只需要請求WDA一次，即可判斷N種文本，減少了N-1次網絡請求耗時，提升執行效率。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如圖7：當出現以下彈框時“label='取消'or label='同意並繼續' or label='我知道了'” ，一次請求即可判斷多種彈窗文本是否存在，存在則逐一處理掉。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/26/263e1fb269d0a1a377c66aa36d78e8ca.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖7 各種彈窗情況","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.3.3保持被測應用處在前臺","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"APP在操作過程中，可能因爲某些操作跳出當前APP，因此保證當前被測APP一直在前臺運行是十分有必要的。常見的方案有兩種","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"方法一，","attrs":{}},{"type":"text","text":"直接分析當前所處頁面DOM樹，獲取XPath返回的XCUIElementTypeApplication屬性，此屬性值與被測APP名稱一致則認爲被測APP保持在最前臺。但是當被測應用頁面複雜時，XPath查詢的速度會變慢，嚴重影響任務執行的單步耗時。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"方法二，","attrs":{}},{"type":"text","text":"利用蘋果官方提供的接口activeApplication可直接輸出當時前臺應用的Bundle ID，返回的Bundle ID與被測應用一致就可以快速確認當前APP是否在前臺。這樣不再查詢頁面整體的XPath信息，避免返回過多的無用信息，也更穩定、快速。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不過方法二也不能處理所有的情況，比如當手機打開了輔助功能（小白點）的面板、或者存在左上角的跳轉返回其他應用按鈕時，方法2會誤判頂層APP爲非被測APP。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"判斷下發的事件是否有效執行","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"穩定性任務運行過程中，如果遇到靜態頁面，在點擊退出之前，當前頁面觸發的操作對於APP而言都是無價值的，我們稱之爲無效點擊，爲了儘量減少無效點擊的數量，我們希望在操作事件之後進行判斷，最直接的想法就是判斷操作之後頁面是否發生了變化。因此需要在事件操作前後進行截圖，將截圖直接生成均值hash的指紋作爲圖像的標記，連續2步驟之間的指紋漢明距離小於閾值則判斷爲同一頁面 (圖8)。連續2步驟相同則判定這個事件無效。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/cf/cf82c1057c18d0a38f7a3f199f0a0132.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":" 圖8 判斷屏幕截圖的相似度","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"解決了測試事件的生成、處理好突發的情況，就可以按下述圖9 服務框架循環執行測試服務。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c4/c47d453d25fe4b62843b17d9dadba6a9.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖9 服務框架","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"二、落地應用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了驗證不同策略下的效果，我們進行了對比實驗，每種策略單獨運行20次，每次連續執行8小時，選取了事件總數、無效佔比、單步耗時和頁面覆蓋數4個指標進行評價。對比數據見表10。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3e/3e2924c6d1da65d436421964cd07f9f4.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"表10 各策略數據對比表格","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在頁面覆蓋數方面，由於iOS端無法獲取類似Android的Activity個數來統計，我們使用近似的數據來代替，利用測試過程中的事件截圖，對截圖去重來近似衡量頁面覆蓋數。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"頁面覆蓋數計算規則：每個圖片計算指紋，圖片之間通過指紋計算相似度，相似度過高則只計一頁；累計所有的截圖數量認爲是此次任務的頁面覆蓋數。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下圖11中，X軸是測試執行時長，Y軸是畫面覆蓋數。可以看到8小時執行後新的按元素遍歷策略覆蓋率比隨機點擊策略提升了將近30%。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/22/22b3a2da3cdc8fa66a5f966b0ad31bf3.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖11 各策略覆蓋的畫面數隨時間增長趨勢圖","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從幾組數據來看，隨機點擊的平均單步耗時短、點擊頻率高，但是有效時間佔比偏低。元素遍歷策略在有效事件佔比和頁面覆蓋數方面提升明顯，但由於需要大量切圖等操作，單步耗時較高。兩種方式各有利弊，可以根據實際的測試場景進行選擇，如果希望驗證APP的抗壓性可以選擇隨機策略，如果希望覆蓋到更多的頁面則可以選擇元素遍歷策略。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.1獨立APP運行","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當前兩種測試均在愛奇藝內使用，穩定性測試任務每月運行800+次，發現崩潰200+，在整個 DevOps 流程中扮演重要的基礎服務角色。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.2模塊指定頁面執行","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除了在獨立APP中運行，也有團隊需要在指定的模塊頁面內開展穩定性測試，我們嘗試在被測頁面DOM樹加入隱藏標識，週期性判斷標識是否跳出了限定範圍。該方案基本滿足業務場景，可持續在限制頁面範圍內執行測試。但對程序也有一定侵入性，需要開發在頁面增加定製化的屬性，供測試工具判斷是否跳出限定範圍。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"三、後續優化","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當前的穩定性測試策略都是蠻力點擊和遍歷，後續會拓展更多的更有針對性的遍歷策略，比如追求用戶實際操作情況的一致，嘗試根據線上用戶流量提取用戶行爲，生成更加切合用戶實際的操作行爲；追求bug發現效率，對歷史上更容易出現崩潰的頁面增加遍歷權重等等。","attrs":{}}]}]}