构建高效Presubmit卡点，落地测试左移最佳实践

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"樊登有一节课讲的挺有意思，说中国有个组织叫绩效改进协会，专门研究用技控代替人控的事情。其用麦当劳来举例子，他说麦当劳其实招人标准很低，高中文凭就可以，但是培养出来的人，三五年之后，每一个都是大家争抢的对象。为什么呢？因为麦当劳的标准化做的很好。其中有一个例子是说，麦当劳的汉堡，出炉之后，15分钟之后没卖掉就扔了。大家想想，如果你是领导，你如何让员工遵循这个标准？而麦当劳的解决方案说起来也简单，汉堡做出来之后就放入保温箱，15分钟后保温箱就会报警，通知餐厅经理该批次的汉堡过期了需要丢弃。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"人是组织不确定性的最大来源，让人去记每个汉堡的出炉时间铁定不行。而我们做工程效能，质量保障的，也同样需要思考如何用技术手段消弭不确定性，以确保更好的做组织升维。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"借此，今天就想谈谈如何通过强化Presubmit卡点模式，落地实实在在的测试左移，让提效言之有物。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PS: 针对PR(github叫法)/MR(gitlab叫法)触发的CI检查就是Presubmit模式","attrs":{}}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Presubmit卡点模式可以包含哪些检查？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"单元测试","attrs":{}},{"type":"text","text":"，必须要有。行业内的基本认知，普及度很高。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"静态检查","attrs":{}},{"type":"text","text":"，必须要有。每种语言都有其推荐的检查工具，建议配上。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"集成测试/e2e","attrs":{}},{"type":"text","text":"，强烈建议有。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"系统性能基准测试","attrs":{}},{"type":"text","text":"，如果你在Presubmit阶段，能够针对系统的核心KPI，持续自动化的做性能基准测试，用于辅助Code Review阶段的代码合入。恭喜，你当属于行业领先地位。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"相信单测、静态检查，大部分项目都有，此处不表。本篇想重点谈谈为什么要在Presubmit阶段引入后几种测试类型，以及行业参考。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"为什么要在Presubmit阶段跑集测、e2e、甚至性能基准测试？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"笔者有以下几点认知：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"提前发现回归问题，降低修复成本","attrs":{}},{"type":"text","text":"。Presubmit阶段，是研发的编码阶段，在这个阶段，如果在测试未介入的情况下，就能提供充分、即时的质量反馈，必然可以极大的提高研发迭代效率。要知道很多时候，QA和研发都是在并行工作，手上会有很多事情在排队处理。而如果一个回归缺陷在QA验收阶段才被发现，那可能已经过去一段时间了。研发需要重新将需求拾起，修复并再提测，很可能会浪费很多时间。这种情形下，隐性的成本损耗会非常大。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"保障提测标准，建立和谐产研关系","attrs":{}},{"type":"text","text":"。很多QA团队都会要求研发在提测上，要有一定的质量标准。这很好，但人非圣贤，孰能无过。尤其在时间紧，任务重时，必然会发生研发自测不充分，遗漏低级问题到QA手上的情况。所以，与其主观约定，不如用自动化建立标准。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"强化测试价值，增加曝光度","attrs":{}},{"type":"text","text":" 很多时候，QA同学写了很多自动化，但是业务无感，研发无感。这时候，你会发现，自动化就成了QA同学手里的玩物，不能有效交付。但是如果在Presubmit阶段就充分执行，不断执行，尤其是能早早的检测出bug时，整个团队必然会更加关注集测产物。长此以往，认可度就会比较高。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"测试左移，真正优雅的保障入库代码质量","attrs":{}},{"type":"text","text":"。谈到测试左移，我看到了太多的流程范，意识流。大家很多时候会放大主观能动性，强调尽早的参与到项目早期。这一点没错，但流程还是依赖于人的值守，但人最是喜新善变。针对回归问题，如果能够建立行之有效的检测手段，必然可以极大的降低心智负担。","attrs":{}}]}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"在Presubmit阶段落地复杂测试类型，有哪些挑战？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"好处很多，但落地也非易事。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"被测系统怎么建设？","attrs":{}},{"type":"text","text":" 不管是集成测试还是e2e，被测对象都是较为完整的业务系统。而要在Presubmit阶段执行起来，就需要通过代码，自动构建和部署整套系统。另，业务通常会有多个仓库，多PR/MR同时执行，所以被测环境应该是按需而起，多套并行。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"资源哪里来？","attrs":{}},{"type":"text","text":" 既然是多套环境，必然会涉及到很多资源，资源哪里来，环境如何有效管理？PR合并之后是不是应该自动回收等等问题。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"使用什么样的系统和姿势来构建？","attrs":{}},{"type":"text","text":"  Presubmit阶段对应的是CI系统，而CI的执行必然需要足够的快。毕竟，提个PR，等十几分钟才有结果反馈，有点不雅。笔者经验认为，对于绝大部分的业务，CI要控制在10分钟以内。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其实深入分析，问题还会有很多。但是挑战即机遇，方法总比困难多。只要价值足够大，收获才能足够多。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"业界有哪些可以参考的？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"七牛云","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"七牛云比较早的开始围绕Presubmit阶段建设各种质量反馈，落地测试左移。并且还进一步做了测试覆盖率收集和受影响服务分析,以及devops建设等工作。其使用的方案和工具大部已开源，比较有借鉴价值。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/49/49c6790c93e34c4f2506e6c9ebc13878.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PS:想进一步了解细节的同学，可以搜索 MTSC2020 Topic: 基于云原生的测试左移技术实践 by 储培","attrs":{}}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"开源工具:","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"go语言系统测试覆盖率收集利器 ","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/qiniu/goc","title":"","type":null},"content":[{"type":"text","text":"https://github.com/qiniu/goc","attrs":{}}]}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"go语言依赖分析 ","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/qiniu/go_dep_search","title":"","type":null},"content":[{"type":"text","text":"https://github.com/qiniu/go_dep_search","attrs":{}}]}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"谷歌","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"谷歌建设Presubmit模式的历史由来已久，其代码大仓的工作模式，也让其在这方面多了一些推陈出新。比如通过Machine Learning / Probabilistic Safety来筛选有效的执行用例，减少Flaky tests等。感兴趣的同学可以查看参考链接:","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://testing.googleblog.com/2008/09/presubmit-and-performance.html","title":"","type":null},"content":[{"type":"text","text":"https://testing.googleblog.com/2008/09/presubmit-and-performance.html","attrs":{}}]}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://testing.googleblog.com/2018/09/efficacy-presubmit.html","title":"","type":null},"content":[{"type":"text","text":"https://testing.googleblog.com/2018/09/efficacy-presubmit.html","attrs":{}}]}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"更多工程效能、测开技术、云原生相关讨论欢迎关注: BigCarlJi","attrs":{}}]}]}