UI遍歷中頁面定義和動作事件篩選方法

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Android UI遍歷測試中，除傳統的基於monkey的隨機性測試外，基於模型的測試在測試覆蓋率和可回溯性上表現更好，是目前熱門的研究方向。在基於模型的測試中，對UI頁面的定義和動作事件的篩選是十分重要而基礎的工作。本文將介紹UI頁面定義和動作事件篩選的具體方法。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"技術實現"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本項目使用python。我們可以使用UI Automator來獲取UI界面的層次樹信息。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"View Tree"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"UI頁面其實是一種樹狀結構的數據，稱作view tree，其節點是每一個子view。每個子view一般包含'resource_id'、'scrollable'、'clickable'、'bounds'等信息，我們可以充分利用這些信息，來對頁面進行定義以及動作事件的篩選。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/d5\/d50e4297db5879f6ee2f93ab7fcca17d.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了便於操作，先將view tree樹結構轉成list列表形式，並保存子view的index索引等信息。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"def get_view_list(view_tree):\n view_tree['parent'] = -1\n view_list = []\n view_tree_to_list(0, view_tree, view_list)\n self.last_acc_event['view_list'] = view_list\n return view_list\n\ndef view_tree_to_list(index, view_tree, view_list):\n tree_id = len(view_list)\n view_tree['temp_id'] = tree_id\n bounds = [[-1, -1], [-1, -1]]\n bounds[0][0] = view_tree['bounds'][0]\n bounds[0][1] = view_tree['bounds'][1]\n bounds[1][0] = view_tree['bounds'][2]\n bounds[1][1] = view_tree['bounds'][3]\n width = bounds[1][0] - bounds[0][0]\n height = bounds[1][1] - bounds[0][1]\n view_tree['size'] = \"%d*%d\" % (width, height)\n view_tree['index'] = index\n view_tree['bounds'] = bounds\n view_list.append(view_tree)\n children_ids = []\n for item in range(len(view_tree['children'])):\n child_tree = view_tree['children'][item]\n child_tree['parent'] = tree_id\n view_tree_to_list(item, child_tree, view_list)\n children_ids.append(child_tree['temp_id'])\n view_tree['children'] = children_ids"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"新頁面的定義"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於App裏有feed頁等可無限刷新的頁面，這就需要對頁面進行定義區分，將類似的頁面歸爲一類，避免UI遍歷過程陷入無限循環的狀態。這裏，我們對頁面有效信息提取成文本，並哈希成字符串作爲該頁面的唯一標識符。具體地，我們提取了每一個子view的'class'、'clickable'、'checked'、'scrollable'、'long-clickable'、'text'這些信息，將UI頁面所有的子view信息組成文本，並用md5哈希成字符串。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"def get_state_str(view_list):\n state_str_raw = get_state_str_raw(view_list)\n return md5(state_str_raw)\n\ndef get_state_str_raw(view_list):\n view_signatures = set()\n for view in view_list:\n view_signature = get_view_signature(view)\n if view_signature:\n view_signatures.add(view_signature)\n return \"%s{%s}\" % (self.foreground_activity, \",\".join(sorted(view_signatures)))\n\ndef get_view_signature(view_dict):\n view_text = view_dict['text']\n if view_text is None or len(view_text) > 50:\n view_text = \"None\"\n\n signature = \"[class]%s[text]%s[%s,%s,%s,%s]\" % \\\n (view_dict['class'],\n view_text,\n view_dict['clickable'],\n view_dict['checked'],\n view_dict['scrollable'],\n view_dict['long-clickable']\n )\n return signature\n"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"動作事件篩選"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們遍歷所有的子view，首先去掉'resource_id'爲'android:id\/navigationBarBackground'、'android:id\/statusBarBackground'的導航欄的view，這在遍歷測試中是不需要的。但有的app，它的導航欄view的'resource_id'不是這個，那就需要新加入過濾的內容，或者直接通過頂欄的座標過濾掉導航欄。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"過濾完系統的view事件之後，我們繼續篩選，這裏我們選取了'scrollable'或者'clickable'爲true的子view。當然我們還可以篩選'enabled'、'focusable'等爲true的子view事件，可以根據項目實際需要自行定義選擇。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"def get_possible_input(view_list):\n possible_events = []\n enabled_view_ids = []\n touch_exclude_view_ids = set()\n for view_dict in view_list:\n if view_dict['enabled'] and \\\n view_dict['resource_id'] not in \\\n ['android:id\/navigationBarBackground',\n 'android:id\/statusBarBackground']:\n enabled_view_ids.append(view_dict['temp_id'])\n\n for view_id in enabled_view_ids:\n if view_list[view_id]['scrollable']:\n possible_events.append(ScrollEvent(view=views_list[view_id], direction=\"UP\"))\n possible_events.append(ScrollEvent(view=views_list[view_id], direction=\"DOWN\"))\n possible_events.append(ScrollEvent(view=views_list[view_id], direction=\"LEFT\"))\n possible_events.append(ScrollEvent(view=views_list[view_id], direction=\"RIGHT\"))\n elif view_list[view_id]['clickable']:\n possible_events.append(TouchEvent(view=views_list[view_id]))\n touch_exclude_view_ids.add(view_id)\n # elif views_list[view_id]['enabled'] and \\\n # views_list[view_id]['focusable']:\n # possible_events.append(TouchEvent(view=views_list[view_id]))\n return possible_events"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們如果想優先把'scrollable'的事件放在前面，可以分開篩選。有的時候向下的ScrollEvent事件是不必要的，我們也可以註釋掉。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"for view_id in enabled_view_ids:\n if views_list[view_id]['scrollable']:\n possible_events.append(ScrollEvent(view=views_list[view_id], direction=\"UP\"))\n # possible_events.append(ScrollEvent(view=views_list[view_id], direction=\"DOWN\"))\n possible_events.append(ScrollEvent(view=views_list[view_id], direction=\"LEFT\"))\n possible_events.append(ScrollEvent(view=views_list[view_id], direction=\"RIGHT\"))\n\nfor view_id in enabled_view_ids:\n if views_list[view_id]['clickable']:\n possible_events.append(TouchEvent(view=views_list[view_id]))\n touch_exclude_view_ids.add(view_id)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外，我們還可以過濾一些不必要的子view事件。比如，有的子view的座標bounds超出了UI界面，這些界面外子view是不需要去測試遍歷的。還有的子view的bounds會擠在一小塊的像素內，這些也是不必要的。這裏，我們設置如果bounds上下邊界座標差小於5時，就過濾掉。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"def filter_possible_input(possible_events,origin_dim=[1080, 1920]):\n filter_events = []\n for event in possible_events:\n # 過濾座標爲負的值\n bounds = event.view[\"bounds\"]\n bounds = [bounds[0][0], bounds[0][1], bounds[1][0], bounds[1][1]]\n x_min = max(0, bounds[0])\n y_min = max(0, bounds[1])\n x_max = min(origin_dim[0], bounds[2])\n y_max = min(origin_dim[1], bounds[3])\n if x_min >= x_max or y_min >= y_max:\n continue\n # 更新bounds座標點\n event.view[\"bounds\"] = [[x_min,y_min],[x_max,y_max]]\n\n # 過濾小於5個像素的event\n if (y_max-y_min) < 5:\n pass\n else:\n filter_events.append(event)\n\n return filter_events"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過對頁面的定義和動作事件的篩選，我們可以將不同的頁面進行區分，相似的頁面歸爲一類，篩選有效的event事件。在此基礎上，我們可以構建圖模型，將測試任務變爲對有向圖的遍歷問題，在圖模型上應用不同的算法，比如深度優先遍歷、啓發式搜索、深度學習或者強化學習算法等，對App進行充分的遍歷測試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文轉載自：360技術（ID：qihoo_tech）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/haY-9TBzn-g0g3lckQJ-MQ","title":"xxx","type":null},"content":[{"type":"text","text":"UI遍歷中頁面定義和動作事件篩選方法"}]}]}]}