Python圖片抓取實操

爬取效果如下(別問我爲什麼被分成了兩張還有去不掉的水印，不然網站怎麼賺錢)：

爬取鏈接爲（千圖網的圖片，順便吐槽一下，這個有些圖片是分兩半的！！！）：

https://www.58pic.com/piccate/10-0-0-p1.html

記得先F12看所需要的內容具體位置和代碼，可以找出，圖片列表需要的href="//www.58pic.com/sucai/19381028.html"：

<a href="//www.58pic.com/sucai/19381028.html" class="thumb-box" data-id="19381028" target="_blank">
<div class="card-trait"><div class="card-tag-business a190"></div></div><img src="//preview.qiantucdn.com/58picmark/back_origin_pic/19/38/10/28M58PICdRdmHaG8694atMaRk.JPG!qt324new_nowater" alt="炫彩星空背景" data-i="0" class="back-static" width="324" height-no="432"><div class="card-bg"></div><div class="card-handle"><div class="handle-dum qt-btn btn-green-linear download-page" data-id="19381028" data-yc="1" data-bg="0"><span class="xfdl"><i class="icon-down"></i>免費下載</span></div><div class="handle-fav" data-action="addFav" data-id="19381028"><span></span></div>
</div>
</a>

詳細頁面所需要的圖片鏈接src="//preview.qiantucdn.com/58picmark/back_origin_pic/19/38/10/28M58PICdRdmHaG8694atMaRk.JPG!w1024_small" class="show-area-pic"：

<img src="//preview.qiantucdn.com/58picmark/back_origin_pic/19/38/10/28M58PICdRdmHaG8694atMaRk.JPG!w1024_small" class="show-area-pic" id="show-area-pic" alt="炫彩星空背景" title="炫彩星空背景" width="650">

記得新建一個存儲圖片的文件夾picture。

抓取代碼如下：

import socket
import re
import requests
from bs4 import BeautifulSoup
from lxml import etree

def test_pachong():
    url = r"https://www.58pic.com/piccate/10-0-0-p1.html"
    headers = {'user-agent': 'my-test/0.0.1','Referer':'https://www.58pic.com/piccate/10-0-0-p1.html'}
    html = requests.get(url,headers).text
    soup = BeautifulSoup(html, 'lxml')
    infoData = soup.find_all(name='a', attrs={'class': 'thumb-box'})
    num = len(infoData)
    for j in range(num):
        picinfo = str(infoData[j])
        if '''target="_blank"''' in picinfo:
            rex1 = re.compile("href=\"(.+?)\"")
            pic_url = rex1.findall(picinfo)[0]
            try:
                get_picInfo(pic_url)
            except:
                print('異常url是{}'.format(pic_url))
def get_picInfo(pic_url):
    headers = {'user-agent': 'my-test/0.0.1','Referer':'https://www.58pic.com/piccate/10-0-0-p1.html'}
    url = "https:"+pic_url
    html = requests.get(url,headers).text
    soup = BeautifulSoup(html, 'lxml')
    infoData = soup.find_all(name='img',attrs={'class':'show-area-pic'})
    num = len(infoData)
    img_title = ''
    img_info = ''
    for j in range(num):
        info = str(infoData[j])
        rex1 = re.compile("title=\"(.+?)\"")
        rex2 = re.compile("src=\"(.+?)\"")
        infoData1 = rex1.findall(info)[0]
        infoData2 = rex2.findall(info)[0]
        img_url = "http:"+infoData2
        img_title = infoData1+'.jpg'
        print(img_title,'--url--',img_url)
        if num > 1: #這裏確認是圖片被網站分成了兩部分，暫時還沒時間處理能拼接這兩部分圖片
            if j == 0:
                save_img('picture\\'+img_title,requests.get(img_url,headers).content)
            else:
                img_title = infoData1+str(j)+'.jpg'
                save_img('picture\\'+img_title,requests.get(img_url,headers).content)
        else:
           save_img('picture\\'+img_title,requests.get(img_url,headers).content)
test_pachong()

最後就是保存我們通過url拿到的圖片源requests.get(img_url,headers).content，代碼如下：

def save_img(file_name,img):
    '''保存圖片'''
    with open (file_name,'wb') as save_img:
        save_img.write(img)
    print ('正在下載{}'.format(file_name))