爬蟲——練習分頁爬取糗事百科熱圖，保存圖片到本地

• os模塊操作：建文件夾
• re模塊操作：單行模式+非貪婪匹配+分組取值 img_list = re.findall('<div class="thumb">.*?<img src="(.*?)".*?>.*?</div>', page_text, re.S)
• requests操作：圖片地址要拼接+ 保存本地時文件名分割+ content屬性+ wb模式寫

缺點是有點慢

import requests
import re
import os

if not os.path.exists('./img2'):
    os.mkdir('./img2')

start_page = int(input("輸入起始頁碼"))
end_page = int(input("輸入終止頁碼"))

proxies = {
    "https": "218.60.8.99:3129"

}

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
}

# https://www.qiushibaike.com/imgrank/page/2/

base_url = 'https://www.qiushibaike.com/imgrank/page/'

for i in range(start_page, end_page + 1):
    print(i)
    url = base_url + str(i)

    response = requests.get(url=url, proxies=proxies, headers=headers)
    page_text = response.text
    img_list = re.findall('<div class="thumb">.*?<img src="(.*?)".*?>.*?</div>', page_text, re.S)
    for url in img_list:
        img_url = 'https:' + url
        # 持久化處理
        img_data = requests.get(url=img_url, proxies=proxies, headers=headers).content
        imgName = url.split('/')[-1]
        withPath = 'img2/' + imgName

        with open(withPath, 'wb')as f:
            f.write(img_data)
            print(imgName + "寫入成功")