最近遇到個很棘手的問題弄了很久才明白。
網頁http://www.hkexnews.hk/sdw/search/mutualmarket_c.aspx
我想獲取裏面的資料，但是需要選取一個日期，那麼意味着我需要發送一個post包給此頁面。
從而發現了2個隨機參數：VIEWSTATE、EVENTVALIDATION
具體解決辦法如下：
def get_hiddenvalue(url):
request=urllib.request.Request(url)
reponse=urllib.request.urlopen(request)
resu=reponse.read()
html = resu.decode('utf-8') # python3
VIEWSTATE =re.findall(r'<input type="hidden" name="VIEWSTATE" id="VIEWSTATE" value="(.?)" />', html,re.I)
EVENTVALIDATION =re.findall(r'<input type="hidden" name="EVENTVALIDATION" id="EVENTVALIDATION" value="(.?)" />', html,re.I)
return VIEWSTATE[0],EVENTVALIDATION[0]
編寫函數先訪問一次網頁。隨後獲取該值之後再發送post包 。解決！
全部源碼如下：
import requests
import urllib.request
import re
NIAN = '2017'
YUE = '12'
RI = '30'
url = 'http://www.hkexnews.hk/sdw/search/mutualmarket_c.aspx'
def get_hiddenvalue(url):
request=urllib.request.Request(url)
reponse=urllib.request.urlopen(request)
resu=reponse.read()
html = resu.decode('utf-8') # python3
VIEWSTATE =re.findall(r'<input type="hidden" name="VIEWSTATE" id="VIEWSTATE" value="(.?)" />', html,re.I)
EVENTVALIDATION =re.findall(r'<input type="hidden" name="EVENTVALIDATION" id="EVENTVALIDATION" value="(.?)" />', html,re.I)
return VIEWSTATE[0],EVENTVALIDATION[0]
VIEWSTATE, EVENTVALIDATION=get_hiddenvalue(url)
data = {
'EVENTVALIDATION':EVENTVALIDATION,
'VIEWSTATE':VIEWSTATE,
'__VIEWSTATEGENERATOR':'EC4ACD6F',
'btnSearch.x':'23',
'btnSearch.y':'12',
'ddShareholdingDay':NIAN,
'ddShareholdingMonth':YUE,
'ddShareholdingYear':RI,
'today':'20180509'
}
html_post = requests.post(url, data=data)
print(html_post.text)