構建高效Presubmit卡點，落地測試左移最佳實踐

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"樊登有一節課講的挺有意思，說中國有個組織叫績效改進協會，專門研究用技控代替人控的事情。其用麥當勞來舉例子，他說麥當勞其實招人標準很低，高中文憑就可以，但是培養出來的人，三五年之後，每一個都是大家爭搶的對象。爲什麼呢？因爲麥當勞的標準化做的很好。其中有一個例子是說，麥當勞的漢堡，出爐之後，15分鐘之後沒賣掉就扔了。大家想想，如果你是領導，你如何讓員工遵循這個標準？而麥當勞的解決方案說起來也簡單，漢堡做出來之後就放入保溫箱，15分鐘後保溫箱就會報警，通知餐廳經理該批次的漢堡過期了需要丟棄。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"人是組織不確定性的最大來源，讓人去記每個漢堡的出爐時間鐵定不行。而我們做工程效能，質量保障的，也同樣需要思考如何用技術手段消弭不確定性，以確保更好的做組織升維。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"藉此，今天就想談談如何通過強化Presubmit卡點模式，落地實實在在的測試左移，讓提效言之有物。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PS: 針對PR(github叫法)/MR(gitlab叫法)觸發的CI檢查就是Presubmit模式","attrs":{}}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Presubmit卡點模式可以包含哪些檢查？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"單元測試","attrs":{}},{"type":"text","text":"，必須要有。行業內的基本認知，普及度很高。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"靜態檢查","attrs":{}},{"type":"text","text":"，必須要有。每種語言都有其推薦的檢查工具，建議配上。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"集成測試/e2e","attrs":{}},{"type":"text","text":"，強烈建議有。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"系統性能基準測試","attrs":{}},{"type":"text","text":"，如果你在Presubmit階段，能夠針對系統的核心KPI，持續自動化的做性能基準測試，用於輔助Code Review階段的代碼合入。恭喜，你當屬於行業領先地位。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"相信單測、靜態檢查，大部分項目都有，此處不表。本篇想重點談談爲什麼要在Presubmit階段引入後幾種測試類型，以及行業參考。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"爲什麼要在Presubmit階段跑集測、e2e、甚至性能基準測試？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"筆者有以下幾點認知：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"提前發現迴歸問題，降低修復成本","attrs":{}},{"type":"text","text":"。Presubmit階段，是研發的編碼階段，在這個階段，如果在測試未介入的情況下，就能提供充分、即時的質量反饋，必然可以極大的提高研發迭代效率。要知道很多時候，QA和研發都是在並行工作，手上會有很多事情在排隊處理。而如果一個迴歸缺陷在QA驗收階段才被發現，那可能已經過去一段時間了。研發需要重新將需求拾起，修復並再提測，很可能會浪費很多時間。這種情形下，隱性的成本損耗會非常大。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"保障提測標準，建立和諧產研關係","attrs":{}},{"type":"text","text":"。很多QA團隊都會要求研發在提測上，要有一定的質量標準。這很好，但人非聖賢，孰能無過。尤其在時間緊，任務重時，必然會發生研發自測不充分，遺漏低級問題到QA手上的情況。所以，與其主觀約定，不如用自動化建立標準。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"強化測試價值，增加曝光度","attrs":{}},{"type":"text","text":" 很多時候，QA同學寫了很多自動化，但是業務無感，研發無感。這時候，你會發現，自動化就成了QA同學手裏的玩物，不能有效交付。但是如果在Presubmit階段就充分執行，不斷執行，尤其是能早早的檢測出bug時，整個團隊必然會更加關注集測產物。長此以往，認可度就會比較高。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"測試左移，真正優雅的保障入庫代碼質量","attrs":{}},{"type":"text","text":"。談到測試左移，我看到了太多的流程範，意識流。大家很多時候會放大主觀能動性，強調儘早的參與到項目早期。這一點沒錯，但流程還是依賴於人的值守，但人最是喜新善變。針對迴歸問題，如果能夠建立行之有效的檢測手段，必然可以極大的降低心智負擔。","attrs":{}}]}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"在Presubmit階段落地複雜測試類型，有哪些挑戰？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"好處很多，但落地也非易事。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"被測系統怎麼建設？","attrs":{}},{"type":"text","text":" 不管是集成測試還是e2e，被測對象都是較爲完整的業務系統。而要在Presubmit階段執行起來，就需要通過代碼，自動構建和部署整套系統。另，業務通常會有多個倉庫，多PR/MR同時執行，所以被測環境應該是按需而起，多套並行。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"資源哪裏來？","attrs":{}},{"type":"text","text":" 既然是多套環境，必然會涉及到很多資源，資源哪裏來，環境如何有效管理？PR合併之後是不是應該自動回收等等問題。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"使用什麼樣的系統和姿勢來構建？","attrs":{}},{"type":"text","text":"  Presubmit階段對應的是CI系統，而CI的執行必然需要足夠的快。畢竟，提個PR，等十幾分鍾纔有結果反饋，有點不雅。筆者經驗認爲，對於絕大部分的業務，CI要控制在10分鐘以內。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其實深入分析，問題還會有很多。但是挑戰即機遇，方法總比困難多。只要價值足夠大，收穫才能足夠多。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"業界有哪些可以參考的？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"七牛雲","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"七牛雲比較早的開始圍繞Presubmit階段建設各種質量反饋，落地測試左移。並且還進一步做了測試覆蓋率收集和受影響服務分析,以及devops建設等工作。其使用的方案和工具大部已開源，比較有借鑑價值。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/49/49c6790c93e34c4f2506e6c9ebc13878.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PS:想進一步瞭解細節的同學，可以搜索 MTSC2020 Topic: 基於雲原生的測試左移技術實踐 by 儲培","attrs":{}}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"開源工具:","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"go語言系統測試覆蓋率收集利器 ","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/qiniu/goc","title":"","type":null},"content":[{"type":"text","text":"https://github.com/qiniu/goc","attrs":{}}]}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"go語言依賴分析 ","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/qiniu/go_dep_search","title":"","type":null},"content":[{"type":"text","text":"https://github.com/qiniu/go_dep_search","attrs":{}}]}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"谷歌","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"谷歌建設Presubmit模式的歷史由來已久，其代碼大倉的工作模式，也讓其在這方面多了一些推陳出新。比如通過Machine Learning / Probabilistic Safety來篩選有效的執行用例，減少Flaky tests等。感興趣的同學可以查看參考鏈接:","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://testing.googleblog.com/2008/09/presubmit-and-performance.html","title":"","type":null},"content":[{"type":"text","text":"https://testing.googleblog.com/2008/09/presubmit-and-performance.html","attrs":{}}]}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://testing.googleblog.com/2018/09/efficacy-presubmit.html","title":"","type":null},"content":[{"type":"text","text":"https://testing.googleblog.com/2018/09/efficacy-presubmit.html","attrs":{}}]}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"更多工程效能、測開技術、雲原生相關討論歡迎關注: BigCarlJi","attrs":{}}]}]}