寫在前面
根據pypistats統計,tep在pypi的下載量達到了1w,對於純個人研發的一款測試小工具來說,已經算不錯了,要知道HttpRunner也才6w啊。tep可以說是我在接口自動化測試這個領域的技術沉澱,凝結了個人經驗和所見所聞的精華之作,它基於Pytest,借鑑了JMeter、RobotFramework、HttpRunner、京東接口測試平臺等各種優秀自動化設計思想,小小工具,蘊含大大能量。相信它也已經影響了不少人,讓初學者知道Pytest該怎麼玩,讓入門者知道Pytest工程化是什麼樣子,讓熟練者可以參考對照優化代碼。然而當我把tep優化到1.0.0正式版以後,爲什麼卻選擇停止維護呢? 一、 小工具的表達力不夠。當我試圖用tep來描繪更多自動化設計思想時,瞬間感覺到了一絲蒼白,我不一定講的清楚,別人也不一定能夠理解,用代碼來交流始終存在着一定門檻。二、每個人對Pytest使用方式不同 。Pytest本身是測試框架,很多人用它來做二次開發,設計”測試框架“,有好的,有差的,不管白貓黑貓能逮到耗子就是好貓,不管設計的如何,能實現接口自動化項目落地就是好框架。tep要想在這個方向上,建立一套標準,幾乎是不可能的。這不併意味我會就此放棄Pytest,相反,我將致力於Pytest平臺化,從做小工具改爲做測試平臺。 測試平臺具有非常直觀的強大表現力,並且具有工程化的規範性,一看就懂,一用就會,一點就通。測試平臺也是能更好的做技術沉澱的,如果說寫小工具是玩玩而已,那麼開發測試平臺就是認真搞技術了。比如,如何提高Pytest並行執行的效率,我相信測試平臺會比小工具,更能給出一個比較完整的解決方案。下次使用Pytest,也許就不是從tep startproject
開始了,而是docker run
。
作爲歸檔,我也把tep1.0.0正式版的完整教程放在這篇文章了,歡迎大家閱讀。
正式版教程: https://dongfanger.gitee.io/blog/tep.html
正式版源碼: https://github.com/dongfanger/tep
tep簡介
tep
是Try Easy Pytest的首字母縮寫,是一款基於pytest測試框架的測試工具,集成了各種實用的第三方包和優秀的自動化測試設計思想,幫你快速實現自動化項目落地。
快速入門
安裝
pip install tep
Mac用戶建議創建虛擬環境並激活:
python3 -m venv venv
source venv/bin/activate
驗證安裝成功:
tep -V
新建項目
tep startproject demo
帶上-venv
參數,可創建單個項目的Python虛擬環境,並在該項目的虛擬環境中安裝tep:
tep startproject demo -venv
啓動FastAPI示例應用
運行utils/fastapi_mock.py
腳本。
執行示例用例
執行examples/tests/test_login_to_pay_flow.py
用例。
查看日誌
============================= test session starts ==============================
collecting ... collected 1 item
test_login_to_pay_flow.py::test
============================== 1 passed in 0.14s ===============================
Process finished with exit code 0
2022-12-27 15:19:37.223 | INFO | fixtures.fixture_login:produce_expensive_data:15 - ----------------開始登錄----------------
2022-12-27 15:19:37.234 | INFO | utils.http_client:request:37 -
Request URL: http://127.0.0.1:5000/login
Request Method: post
Request Headers: {"Content-Type": "application/json"}
Request Payload: {"json": {"username": "dongfanger", "password": "123456"}}
Status Code: 200
Response: {"token":"de2e3ffu29"}
Elapsed: 0.003s
2022-12-27 15:19:37.234 | INFO | fixtures.fixture_login:produce_expensive_data:23 - ----------------登錄成功----------------
PASSED [100%]2022-12-27 15:19:37.235 | INFO | utils.step:__init__:12 - ----------------搜索商品----------------
2022-12-27 15:19:37.250 | INFO | utils.http_client:request:37 -
Request URL: http://127.0.0.1:5000/searchSku
Request Method: get
Request Headers: {"token": "de2e3ffu29"}
Request Payload: {"params": {"skuName": "\u7535\u5b50\u4e66"}}
Status Code: 200
Response: {"skuId":"222","price":"2.3"}
Elapsed: 0.001s
2022-12-27 15:19:37.250 | INFO | utils.step:__init__:12 - ----------------添加購物車----------------
2022-12-27 15:19:37.254 | INFO | utils.http_client:request:37 -
Request URL: http://127.0.0.1:5000/addCart
Request Method: post
Request Headers: {"token": "de2e3ffu29"}
Request Payload: {"json": {"skuId": "222", "skuNum": 2}}
Status Code: 200
Response: {"skuId":"222","price":"2.3","skuNum":"3","totalPrice":"6.9"}
Elapsed: 0.001s
2022-12-27 15:19:37.254 | INFO | utils.step:__init__:12 - ----------------下單----------------
2022-12-27 15:19:37.257 | INFO | utils.http_client:request:37 -
Request URL: http://127.0.0.1:5000/order
Request Method: post
Request Headers: {"token": "de2e3ffu29"}
Request Payload: {"json": {"orderId": 222, "payAmount": "0.2", "skuId": "222", "price": "2.3", "skuNum": "3", "totalPrice": "6.9"}}
Status Code: 200
Response: {"orderId":"333"}
Elapsed: 0.001s
2022-12-27 15:19:37.257 | INFO | utils.step:__init__:12 - ----------------支付----------------
2022-12-27 15:19:37.259 | INFO | utils.http_client:request:37 -
Request URL: http://127.0.0.1:5000/pay
Request Method: post
Request Headers: {"token": "de2e3ffu29"}
Request Payload: {"json": {"skuId": 123, "price": 0.1, "skuNum": 2, "totalPrice": 0.2, "orderId": "333"}}
Status Code: 200
Response: {"success":"true"}
Elapsed: 0.001s
能在本地跑起來看到日誌且沒有報錯,恭喜您,上手成功!
目錄結構說明
examples:示例代碼,可無顧慮全刪;
fixtures:Pytest fixture,自動導入;
resources:環境變量、全局變量;
tests:測試用例;
utils:工具包;
conftest.py:Pytest掛載;
pytest.ini:Pytest配置;
reports:測試報告,默認不顯示,生成報告後會出現;
用例組織形式
推薦MVC分層設計和數據代碼分離。
小提示:tep老版本的極速寫法,即接口、數據、代碼都放在一個文件的一個函數的寫法,仍然適合於新手或追求效率時使用。
用例集
在tests目錄下將測試用例按功能模塊分成多個用例集:
tests
user
teacher
student
測試用例
必須遵循用例解耦原則,每條用例都是單獨可運行的。用例由2個文件組成,一個文件存放純粹的yaml數據,一個文件存放邏輯代碼:
test_login_to_pay_flow.yaml
test_login_to_pay_flow.py
測試數據
存放在yaml文件中,第一層爲說明文字,第二層爲請求json:
"查詢SKU": {
"skuName": "電子書"
}
"添加購物車": {
"skuId": 123,
"skuNum": 2
}
"下單": {
"orderId": 222,
"payAmount": "0.2"
}
"支付": {
"skuId": 123,
"price": 0.1,
"skuNum": 2,
"totalPrice": 0.2
}
測試標題
測試標題採用了@allure.title("")
:
@allure.title("從登錄到下單支付")
def test(login, env_vars, case_vars):
測試步驟
一條測試用例由多個測試步驟組成:
@allure.title("從登錄到下單支付")
def test(login, env_vars, case_vars):
case_vars.put("token", login["token"])
cache = TepCache(env_vars=env_vars, case_vars=case_vars)
Step("搜索商品", step_search_sku, cache)
Step("添加購物車", step_add_cart, cache)
Step("下單", step_order, cache)
Step("支付", step_pay, cache)
Step第一個參數爲步驟描述,第二個參數爲步驟實現函數,第三個參數爲cache緩存。
步驟實現函數定義在用例文件中:
def step_add_cart(cache: TepCache):
url = cache.env_vars["domain"] + "/addCart"
headers = {"token": cache.case_vars.get("token")}
body = data("添加購物車")
body["skuId"] = cache.case_vars.get("skuId")
response = request("post", url=url, headers=headers, json=body)
assert response.status_code < 400
cache.case_vars.put("skuNum", response.jsonpath("$.skuNum"))
cache.case_vars.put("totalPrice", response.jsonpath("$.totalPrice"))
從上往下依次爲url、headers、body、參數化、請求調用、斷言、數據提取。
小技巧:打開PyCharm的Show Members,就能快速定位到某個step函數,編輯測試步驟。
變量
環境變量:在resources/env_vars
下預填變量,在resources/tep.yaml
中激活某個環境,在代碼中引入env_vars fixture讀取變量值:
def test(env_vars):
logger.info(env_vars["domain"])
全局變量:在resources/global_vars.yaml
預填變量,在代碼中引入global_vars fixture讀取變量值:
def test(global_vars):
print(global_vars["desc"])
用例變量:在用例中引入case_vars fixture,在步驟函數間通過cache傳遞:
@allure.title("從登錄到下單支付")
def test(login, env_vars, case_vars):
case_vars.put("token", login["token"])
cache = TepCache(env_vars=env_vars, case_vars=case_vars)
Step("搜索商品", step_search_sku, cache)
Step("添加購物車", step_add_cart, cache)
Step("下單", step_order, cache)
Step("支付", step_pay, cache)
def step_search_sku(cache: TepCache):
url = cache.env_vars["domain"] + "/searchSku"
headers = {"token": cache.case_vars.get("token")}
body = data("查詢SKU")
response = request("get", url=url, headers=headers, params=body)
assert response.status_code < 400
cache.case_vars.put("skuId", response.jsonpath("$.skuId"))
cache.case_vars.put("skuPrice", response.jsonpath("$.price"))
def step_add_cart(cache: TepCache):
url = cache.env_vars["domain"] + "/addCart"
headers = {"token": cache.case_vars.get("token")}
body = data("添加購物車")
body["skuId"] = cache.case_vars.get("skuId")
response = request("post", url=url, headers=headers, json=body)
assert response.status_code < 400
cache.case_vars.put("skuNum", response.jsonpath("$.skuNum"))
cache.case_vars.put("totalPrice", response.jsonpath("$.totalPrice"))
接口關聯
如上所述,通過case_vars和cache實現了步驟函數裏面的接口關聯,上一個接口的響應,提取後存入cache,下一個接口的入參,從cache取值。
數據提取
utils/http_client.py
封裝了requests.Response,添加了jsonpath方法,支持簡單取值:
response.jsonpath("$.skuNum")
斷言
採用Python原生的assert斷言。16種常用斷言如下:
import allure
@allure.title("等於")
def test_assert_equal():
assert 1 == 1
@allure.title("不等於")
def test_assert_not_equal():
assert 1 != 2
@allure.title("大於")
def test_assert_greater_than():
assert 2 > 1
@allure.title("小於")
def test_assert_less_than():
assert 1 < 2
@allure.title("大於等於")
def test_assert_less_or_equals():
assert 2 >= 1
assert 2 >= 2
@allure.title("小於等於")
def test_assert_greater_or_equals():
assert 1 <= 2
assert 1 <= 1
@allure.title("長度相等")
def test_assert_length_equal():
assert len("abc") == len("123")
@allure.title("長度大於")
def test_assert_length_greater_than():
assert len("hello") > len("123")
@allure.title("長度小於")
def test_assert_length_less_than():
assert len("hi") < len("123")
@allure.title("長度大於等於")
def test_assert_length_greater_or_equals():
assert len("hello") >= len("123")
assert len("123") >= len("123")
@allure.title("長度小於等於")
def test_assert_length_less_or_equals():
assert len("123") <= len("hello")
assert len("123") <= len("123")
@allure.title("字符串相等")
def test_assert_string_equals():
assert "dongfanger" == "dongfanger"
@allure.title("以...開頭")
def test_assert_startswith():
assert "dongfanger".startswith("don")
@allure.title("以...結尾")
def test_assert_startswith():
assert "dongfanger".endswith("er")
@allure.title("正則匹配")
def test_assert_regex_match():
import re
assert re.findall(r"don.*er", "dongfanger")
@allure.title("包含")
def test_assert_contains():
assert "fang" in "dongfanger"
assert 2 in [2, 3]
assert "x" in {"x": "y"}.keys()
@allure.title("類型匹配")
def test_assert_type_match():
assert isinstance(1, int)
assert isinstance(0.2, float)
assert isinstance(True, bool)
assert isinstance(3e+26j, complex)
assert isinstance("hi", str)
assert isinstance([1, 2], list)
assert isinstance((1, 2), tuple)
assert isinstance({"a", "b", "c"}, set)
assert isinstance({"x": 1}, dict)
測試報告
allure下載地址: https://github.com/allure-framework/allure2/releases
解壓後將bin目錄添加到系統環境變量Path。
在pytest命令行添加參數--tep-reports
就能一鍵生成Allure測試報告,並且會把請求入參和響應出參,記錄在測試報告中。
pytest --tep-reports
若想在資源管理器中打開,需要執行命令allure open 報告所在文件夾名
才能正常打開。
用例執行
串行
使用pytest
命令即可。
並行
使用pytest -n auto
,由pytest-xdist提供支持。
特色功能
fixtures自動導入
不是必須在conftest.py裏面定義fixture。只要在fixtures目錄下,創建以fixture_
開頭的文件,fixture都會自動加載到pytest中,方便管理維護。
全局執行一次登錄
預置了fixtures/fixture_login.py
登錄接口,且全局僅執行一次,解決token複用問題:
import pytest
from loguru import logger
from utils.http_client import request
@pytest.fixture(scope="session")
def login(tep_context_manager, env_vars):
"""
tep_context_manager是爲了兼容pytest-xdist分佈式執行的上下文管理器
該login只會在整個運行期間執行一次
"""
def produce_expensive_data(variable):
logger.info("----------------開始登錄----------------")
response = request(
"post",
url=variable["domain"] + "/login",
headers={"Content-Type": "application/json"},
json={"username": "dongfanger", "password": "123456"}
)
assert response.status_code < 400
logger.info("----------------登錄成功----------------")
return response.json()
return tep_context_manager(produce_expensive_data, env_vars)
即便在xdist分佈式場景下,也只會執行一次登錄。
工具包
cache.py,提供緩存。
class TepCache:
"""
提供緩存服務,包括全局變量、環境變量、用例變量
"""
def __init__(self, global_vars=None, env_vars=None, case_vars=None):
self.global_vars = global_vars
self.env_vars = env_vars
self.case_vars = case_vars
dao.py,目前支持訪問MySQL,需要安裝pymysql、sqlalchemy和pandas。
#!/usr/bin/python
# encoding=utf-8
"""
@Author : dongfanger
@Date : 9/2/2020 11:32 AM
@Desc : 訪問數據庫
"""
from loguru import logger
try:
from sqlalchemy import create_engine
from texttable import Texttable
except ModuleNotFoundError:
pass
def mysql_engine(host, port, user, password, db):
try:
engine = create_engine(f"mysql+pymysql://{user}:{password}@{host}:{port}/{db}")
except NameError:
return ""
return engine
def print_db_table(data_frame):
"""以表格形式打印數據表"""
tb = Texttable()
tb.header(data_frame.columns.array)
tb.set_max_width(0)
# text * cols
tb.set_cols_dtype(['t'] * data_frame.shape[1])
tb.add_rows(data_frame.to_numpy(), header=False)
logger.info(tb.draw())
fastapi_mock.py,示例應用。
#!/usr/bin/python
# encoding=utf-8
import uvicorn
from fastapi import FastAPI, Request
app = FastAPI()
@app.post("/login")
async def login(req: Request):
body = await req.json()
if body["username"] == "dongfanger" and body["password"] == "123456":
return {"token": "de2e3ffu29"}
return ""
@app.get("/searchSku")
def search_sku(req: Request):
if req.headers.get("token") == "de2e3ffu29" and req.query_params.get("skuName") == "電子書":
return {"skuId": "222", "price": "2.3"}
return ""
@app.post("/addCart")
async def add_cart(req: Request):
body = await req.json()
if req.headers.get("token") == "de2e3ffu29" and body["skuId"] == "222":
return {"skuId": "222", "price": "2.3", "skuNum": "3", "totalPrice": "6.9"}
return ""
@app.post("/order")
async def order(req: Request):
body = await req.json()
if req.headers.get("token") == "de2e3ffu29" and body["skuId"] == "222":
return {"orderId": "333"}
return ""
@app.post("/pay")
async def pay(req: Request):
body = await req.json()
if req.headers.get("token") == "de2e3ffu29" and body["orderId"] == "333":
return {"success": "true"}
return ""
if __name__ == '__main__':
uvicorn.run("fastapi_mock:app", host="127.0.0.1", port=5000)
func.py,常用函數,比如pairwise自動生成用例等。
#!/usr/bin/python
# encoding=utf-8
"""
@Author : dongfanger
@Date : 7/24/2020 5:41 PM
@Desc : tep函數庫
"""
import copy
import inspect
import itertools
import json
import os
import time
from sys import stdout
import yaml
from loguru import logger
from utils.project import Project
def current_time():
"""
當前時間,年-月-日 時-分-秒
:return:
"""
return time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(time.time()))
def current_date():
"""
當前日期 年-月-日
:return:
"""
return time.strftime("%Y-%m-%d", time.localtime(time.time()))
def print_progress_bar(i):
"""
進度條
"""
c = int(i / 10)
progress = '\r %2d%% [%s%s]'
a = '■' * c
b = '□' * (10 - c)
msg = progress % (i, a, b)
stdout.write(msg)
stdout.flush()
def case_pairwise(option):
"""
pairwise算法
"""
cp = [] # 笛卡爾積
s = [] # 兩兩拆分
for x in eval('itertools.product' + str(tuple(option))):
cp.append(x)
s.append([i for i in itertools.combinations(x, 2)])
logger.info('笛卡爾積:%s' % len(cp))
del_row = []
print_progress_bar(0)
s2 = copy.deepcopy(s)
for i in range(len(s)): # 對每行用例進行匹配
if (i % 100) == 0 or i == len(s) - 1:
print_progress_bar(int(100 * i / (len(s) - 1)))
t = 0
for j in range(len(s[i])): # 對每行用例的兩兩拆分進行判斷,是否出現在其他行
flag = False
for i2 in [x for x in range(len(s2)) if s2[x] != s[i]]: # 找同一列
if s[i][j] == s2[i2][j]:
t = t + 1
flag = True
break
if not flag: # 同一列沒找到,不用找剩餘列了
break
if t == len(s[i]):
del_row.append(i)
s2.remove(s[i])
res = [cp[i] for i in range(len(cp)) if i not in del_row]
logger.info('過濾後:%s' % len(res))
return res
def load_yaml(path: str) -> dict:
with open(path, encoding="utf8") as f:
return yaml.load(f.read(), Loader=yaml.FullLoader)
def jwt_headers(token):
"""
jwt請求頭
"""
return {"Content-Type": "application/json", "authorization": f"Bearer {token}"}
def data(first_node: str) -> dict:
"""
讀用例同名的yaml文件
取首節點的值
"""
caller = inspect.stack()[1]
case_path = os.path.dirname(caller.filename)
basename = os.path.basename(caller.filename)
data_path_yml = os.path.join(case_path, basename.rstrip(".py") + ".yml")
data_path_yaml = os.path.join(case_path, basename.rstrip(".py") + ".yaml")
node_value = {}
if not os.path.exists(data_path_yml) and not os.path.exists(data_path_yaml):
logger.error("數據文件不存在")
return node_value
data_path = data_path_yml if os.path.exists(data_path_yml) else data_path_yaml
try:
return load_yaml(data_path)[first_node]
except KeyError:
logger.error(f"數據文件{data_path}不存在首節點{first_node}")
http_client.py,requests庫的猴子補丁,可自定義。
#!/usr/bin/python
# encoding=utf-8
import decimal
import json
import time
import allure
import jsonpath
import requests
import urllib3
from loguru import logger
from requests import Response
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
def request(method, url, **kwargs):
template = """\n
Request URL: {}
Request Method: {}
Request Headers: {}
Request Payload: {}
Status Code: {}
Response: {}
Elapsed: {}
"""
start = time.process_time()
response = requests.request(method, url, **kwargs) # requests.request原生用法
end = time.process_time()
elapsed = str(decimal.Decimal("%.3f" % float(end - start))) + "s"
headers = kwargs.get("headers", {})
kwargs.pop("headers")
payload = kwargs
log = template.format(url, method, json.dumps(headers), json.dumps(payload), response.status_code, response.text,
elapsed)
logger.info(log)
allure.attach(log, f'request & response', allure.attachment_type.TEXT)
return TepResponse(response)
class TepResponse(Response):
"""
二次封裝requests.Response,添加額外方法
"""
def __init__(self, response):
super().__init__()
for k, v in response.__dict__.items():
self.__dict__[k] = v
def jsonpath(self, expr):
"""
此處強制取第一個值,便於簡單取值
如果複雜取值,建議直接jsonpath原生用法
"""
return jsonpath.jsonpath(self.json(), expr)[0]
mitm.py,流量錄製,做的不是很好,將就看看。
#!/usr/bin/python
# encoding=utf-8
# mitmproxy錄製流量自動生成用例
import os
import time
from mitmproxy import ctx
project_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
tests_dir = os.path.join(project_dir, "tests")
# tests/mitm
mitm_dir = os.path.join(tests_dir, "mitm")
if not os.path.exists(mitm_dir):
os.mkdir(mitm_dir)
# 當前時間作爲文件名
filename = f'test_{time.strftime("%Y%m%d_%H%M%S", time.localtime())}.py'
case_file = os.path.join(mitm_dir, filename)
# 生成用例文件
template = """import allure
from utils.http_client import request
@allure.title("")
def test(env_vars):
"""
if not os.path.exists(case_file):
with open(case_file, "w", encoding="utf8") as fw:
fw.write(template)
class Record:
def __init__(self, domains):
self.domains = domains
def response(self, flow):
if self.match(flow.request.url):
# method
method = flow.request.method.lower()
ctx.log.error(method)
# url
url = flow.request.url
ctx.log.error(url)
# headers
headers = dict(flow.request.headers)
ctx.log.error(headers)
# body
body = flow.request.text or {}
ctx.log.error(body)
with open(case_file, "a", encoding="utf8") as fa:
fa.write(self.step(method, url, headers, body))
def match(self, url):
if not self.domains:
ctx.log.error("必須配置過濾域名")
exit(-1)
for domain in self.domains:
if domain in url:
return True
return False
def step(self, method, url, headers, body):
if method == "get":
body_grammar = f"params={body}"
else:
body_grammar = f"json={body}"
return f"""
# 描述
# 數據
# 請求
response = request(
"{method}",
url="{url}",
headers={headers},
{body_grammar}
)
# 提取
# 斷言
assert response.status_code < 400
"""
# ==================================配置開始==================================
addons = [
Record(
# 過濾域名
[
"http://www.httpbin.org",
"http://127.0.0.1:5000"
],
)
]
# ==================================配置結束==================================
"""
==================================命令說明開始==================================
# 正向代理(需要手動打開代理)
mitmdump -s mitm.py
# 反向代理
mitmdump -s mitm.py --mode reverse:http://127.0.0.1:5000 --listen-host 127.0.0.1 --listen-port 8000
==================================命令說明結束==================================
"""
project.py,項目基本信息,比如根目錄路徑。
import os
class Project:
root_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
data_dir = os.path.join(root_dir, "data")
step.py,測試步驟的泛化調用。
from loguru import logger
from utils.cache import TepCache
class Step:
"""
測試步驟,泛化調用
"""
def __init__(self, name: str, action, cache: TepCache):
logger.info("----------------" + name + "----------------")
action(cache)
關於tep的更多技術細節,請在源碼中一探究竟吧。也可以添加微信cekaigang,隨時與我聯繫。
結束與開始
tep小工具發佈了1.0.0正式版,我也將不再對其進行維護。對Pytest做接口測試自動化的探索遠沒有結束,我將從EasyPytest測試平臺開始,繼續研究Pytest框架的自動化落地實踐,那些對測試技術的熱情,終將使我們再次相遇。
EasyPytest平臺開發版: https://gitee.com/dongfanger/easy-pytest
EasyPytest平臺正式版: https://github.com/dongfanger/EasyPytest