基礎篇之requests及編碼

1、requests模塊

import requests
from fake_useragent  import UserAgent
headers={"Referer":"http://www.angelimg.com"}
url ="http://image.angelimg.com/00000mx00000/9iM1QyCDV5QJQ5xz2cMC307208/u5utBb7buzfQnHaoSP3u307208-juEeGN.jpg"
# requests模塊get或post方法中的strem參數默認是Flase，如果爲True表示從服務器獲取套接字響應
response = requests.get(url,headers=headers,stream=True)
# requests.get()或者requests.post()方法默認全部下載內存中，下載完成才存在硬盤，可以用Response.iter_content()方法邊下載邊存儲到硬盤
response.raise_for_status() # 請求錯誤時候raise_for_status()會拋出異常，正常會是返回結果是None
with open("a.jpg","wb") as f:
    for chunk in response.iter_content(chunk_size=1024): # chunk_size表示字節數,這裏1024個字節也就是1k大小
        f.write(chunk)
        f.flush()

注意以上適合處理下載大文件操作。

 

 

 

 

【requests常見異常】 

requests.exceptions.SSLError
requests.exceptions.ProxyError
requests.exceptions.ConnectTimeout
requests.exceptions.Timeout
requests.exceptions.HTTPError
requests.exceptions.ConnectTimeout
requests.exceptions.ReadTimeout

2、編碼

【1】例子把“\u5408\u5ddd”轉漢字

city_domain =city_info[0].encode("utf-8").decode("unicode_escape")
city_name = city_info[1].encode("utf-8").decode("unicode_escape")

【2】把字體加密取出爲&#x開頭

 code_list = font.getGlyphOrder()[2:]
 new_list = [code.replace('uni', '\\u') for code in code_list]
 print('替換之後', new_list)
 text = ''.join(new_list)
    # print(text)
 text = text.encode('utf-8').decode('unicode_escape')

 注意：這種加密的字體儘量採用正則表達式提取

        self.browser.get("https://maoyan.com/films/1212608")
        time.sleep(3)
        html = Selector(self.browser.page_source)
        # box_office是一個parsel.selector.SelectorList對象，這個對象是列表類型
        box_office = html.xpath("//p[text()='累計票房']/following-sibling::div[1]/span[@class='stonefont']")
        # box_office[0]是正文對應的Selector對象，Selector對象的root屬性獲得整個網頁結果信息，頁面結構信息是一個HtmlElement對象，etree.tostring()返回是一個二進制類型
        piaofang = etree.tostring(box_office[0].root).decode()
        print(piaofang) #獲取頁面源碼 <span class="stonefont">&#59722;&#61500;</span>

【例子】

html_str = r'\u8fd8\u6ca1\u6709\u4eba\u8bc4\u8bba\uff0c\u8d76\u5feb\u62a2\u4e2a\u6c99\u53d1'
print(html_str.replace('\/', '/').encode().decode('unicode-escape')) #還沒有人評論，趕快搶個沙發