Python爬蟲實戰，requests模塊，爬蟲採集網易財經股票交易數據

前言

本文給大家分享的是如何通過Python爬蟲採集網易財經易數據

開發工具

Python版本： 3.8

相關模塊：

requests模塊
parsel模塊

環境搭建

安裝Python並添加到環境變量，pip安裝需要的相關模塊即可。

思路分析

本文以爬蟲網易財經數據，講解如何採集股票歷史交易數據

爬取網易財經數據如下圖所示:

提取頁面數據

瀏覽器中打開我們要爬取的頁面
按F12進入開發者工具，查看我們想要的數據在哪裏
這裏我們需要股票頁面數據就可以了

代碼實現

    # 1. 發送請求
    response = requests.get(url)
    # 2. 獲取數據
     json_data = response.json()
    # 3. 解析數據
    list = json_data['list']   # 所有的股票信息
        for i in list:
        # i 代表的是每一個股票的 信息
        SYMBOL = i['SYMBOL']
        NAME = i['NAME']
        PRICE = i['PRICE']
        PERCENT = i['PERCENT']
        UPDOWN = i['UPDOWN']
        FIVE_MINUTE = i['FIVE_MINUTE']
        OPEN = i['OPEN']
        YESTCLOSE = i['YESTCLOSE']
        HIGH = i['HIGH']
        LOW = i['LOW']
        VOLUME = i['VOLUME']
        TURNOVER = i['TURNOVER']
        HS = i['HS']
        try:
            LB = i['LB']
        except:
            LB = ""
        WB = i['WB']
        ZF = i['ZF']
        try:
            PE = i['PE']
        except:
            PE = ""
        TCAP = i['TCAP']
        MFSUM = i['MFSUM']
        MFRATIO2 = i['MFRATIO']['MFRATIO2']
        MFRATIO10 = i['MFRATIO']['MFRATIO10']
        print(SYMBOL, NAME, PRICE, PERCENT, UPDOWN, FIVE_MINUTE, OPEN, YESTCLOSE, HIGH, LOW, VOLUME, TURNOVER, HS, LB, WB, ZF, PE, TCAP, MFSUM, MFRATIO2, MFRATIO10)
        # 4. 保存數據
        csv_writer.writerow([SYMBOL, NAME, PRICE, PERCENT, UPDOWN, FIVE_MINUTE, OPEN, YESTCLOSE, HIGH, LOW, VOLUME, TURNOVER, HS, LB, WB, ZF, PE, TCAP, MFSUM, MFRATIO2, MFRATIO10])

結果展示

圖片僅供參考