Python爬蟲實戰：使用代理（requests模塊）

對於requests來說，代理設置加簡單，我們只需要傳入proxies參數即可

import requests

url = 'http://ip.tool.chinaz.com/'

headers = {
    'Host': "ip.tool.chinaz.com",
    'User-Agent': "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36"
}
proxy = '58.240.53.196:8080'

proxy_support = {
    'http':'http://'+proxy,
    'https':'https://'+proxy,
}

response = requests.get(url,proxies=proxy_support)
print(response.text)

如果代理需要認證，同樣在代理的前面加上用戶名密碼即可

proxies='username:[email protected]:9743'