Python抓取靜態網頁

靜態網頁抓取

安裝Requests

pip install requests

獲取響應內容

import requests
# 定製參數
key_dict = {
    'key1': 'value1',
    'key2': 'value2'
}
# 定製請求頭header
headers = {
    'user-agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x63) AppleWebKit/5……',
    'Host': 'www.santostrang.com'
}
response = requests.get('http://www.santostang.com/', params=key_dict, headers=headers)
print('文本編碼：', response.encoding)
print('響應狀態碼：', response.status_code)
print('字符串方式的響應體：', response.text)

發送POST請求

#添加一個data參數
response = requests.get('http://httpbin.org/post', data=key_dict)

設置超時

#添加timeout參數
response = requests.get(link, timeout=0.01)