使用Requests+xpath實現簡單的數據爬取

最近又重新爬蟲，遇到一個之前沒注意的問題，就是request返回對象的問題

from lxml import etree        

currentAllInfo = []
        headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0'}
        sourceHtml = requests.get(url, headers=headers)//sourceHtml對象類型爲response，不是                htmlresponse，因此需要轉換

        selector = etree.HTML(sourceHtml)
        titles = selector.xpath('//dl[@class="blog_list clearfix"]//dd')
        singlePageInfo = {};
        for vs in titles:
            info = vs.xpath('h3[@class="tracking-ad"]/a/text()')
            print "標題:" + info[0]
            singlePageInfo['title'] = info[0]
            time = vs.xpath('div[@class="blog_list_b clearfix"]/div[@class="blog_list_b_r fr"]/label/text()')
            print "時間:" + time[0]
            singlePageInfo['time'] = time[0]
            readCount = vs.xpath('div[@class="blog_list_b clearfix"]/div[@class="blog_list_b_r fr"]/span/em/text()')
            print "閱讀次數:" + readCount[0]
            currentAllInfo.append(singlePageInfo)
        print currentAllInfo