selenium攻佔煎蛋妹子圖

python 版本 3.5
依賴庫：

selenium
beautifulsoap4
lxml
requests

使用selenium的原因是
requests庫 在實際操作的時候，發現請求返回的內容裏面並沒有圖片的鏈接：

<p> <img src="//img.jandan.net/img/blank.gif" onload="jandan_load_img(this)"/> <span class="img-hash">6fadiP6jpEOinbyOjDMf5F1MT01mhMHpB0oC562st3bqZwhPR+OhO+YvbyrNqKyKMmBNGSDh7Gk0I+B+zcKmrgCm3n1M0bXlNjhOjdDps9/hCO039Uo2+w</span> </p>

而在chrome中使用F12去可以看到圖片鏈接，看來這個頁面需要調用js方法纔會返回圖片的url

那我們 只好使用 Selenium+Phantomjs 來實現需求

第一步

導入依賴庫：

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import requests
from bs4 import BeautifulSoup

第二步

使用Selenium+Phantomjs爬取完整的頁面源碼,用bs解析網頁，定位元素

path='D:/jd_image/'
urls = ["http://jandan.net/ooxx/page-{}#comments".format(str(i)) for i in range(1, 50)]
img_url=[]
driver = webdriver.PhantomJS(executable_path=r'D:\phantomjs-2.1.1-windows\bin\phantomjs.exe')
for url in urls:
    driver.get(url)
    data = driver.page_source
    soup = BeautifulSoup(data, "lxml")  #解析網頁
    images = soup.select("a.view_img_link")  #定位元素
    for i in images:               
        z=i.get('href')
        if str('gif') in str(z):
           pass
        else:
            http_url = "http:" + z
            img_url.append(http_url)
            #print("http:%s" % z)

第三步

下載圖片

for j in img_url:
        r=requests.get(j)
        print('正在下載 %s......' % j)
        with open(path+j[-15:],'wb')as jpg:
            jpg.write(r.content)

完整代碼：

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import requests
from bs4 import BeautifulSoup

path='D:/jd_image/'
urls = ["http://jandan.net/ooxx/page-{}#comments".format(str(i)) for i in range(1, 50)]
img_url=[]
driver = webdriver.PhantomJS(executable_path=r'D:\phantomjs-2.1.1-windows\bin\phantomjs.exe')

for url in urls:
    driver.get(url)
    data = driver.page_source
    soup = BeautifulSoup(data, "lxml")
    images = soup.select("a.view_img_link")

    for i in images:               
        z=i.get('href')
        if str('gif') in str(z):
           pass
        else:
            http_url = "http:" + z
            img_url.append(http_url)
            #print("http:%s" % z)

    for j in img_url:
        r=requests.get(j)
        print('正在下載 %s......' % j)
        with open(path+j[-15:],'wb')as jpg:
            jpg.write(r.content)