使用python3模擬http頭部信息

有的網站爲了防止惡意爬取數據，對UA頭進行了校驗，判斷是否是來自於瀏覽器的請求，我們使用python3請求的UA頭是 “User-Agent”: “Python-urllib/3.5”，只要將UA頭指定爲瀏覽器UA頭或者任意內容。

root@VM-0-7-ubuntu:~/python/zeropython3/15# cat post_4.py 
from urllib import parse
from urllib import request

headers={
"User-Agent":"I am UA!"
}

data = bytes(parse.urlencode({'hello':'world'}),encoding = 'utf8')

req = request.Request('http://httpbin.org/post',data=data,headers=headers,method = 'POST')
response = request.urlopen(req)
print (response.read().decode('utf-8'))
------------------------------------------------------------------------------------------
root@VM-0-7-ubuntu:~/python/zeropython3/15# python3 post_4.py 
{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {
    "hello": "world"
  }, 
  "headers": {
    "Accept-Encoding": "identity", 
    "Content-Length": "11", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "httpbin.org", 
    "User-Agent": "I am UA!"  #成功替換
  }, 
  "json": null, 
  "origin": "xxxxx, xxxxxxx", 
  "url": "https://httpbin.org/post"
}