前面 《Python抓取CSV文件中数据》是通过Python 的csv模块直接调用。但这周自己在做一个one-person project的时候,发现这样后面数据管理会有些不方便,就转而选择数据库pysql了(安装:pip install pymysql)。
第一步,安装MySQL,通过Table Data Import wizard导入.csv文件。
第二步,MySQL测试链接。
第三步,Python实现与MySQL的链接:
mycon = pymysql.connect(host='127.0.0.1', user='*******', passwd='******', db='csvimages', port=3306, charset='utf8') mycur =mycon.cursor() mycur.execute('select * from imgs where(`imgs`.`OriginalURL` like "%http%" and Title like "%ic%")')
Test Result :
# -*- coding : utf-8 -*- import pymysql import os import urllib2 import threading class CsvReaderImageTwo(threading.Thread): def __init__(self): threading.Thread.__init__(self) def action(self): try: mycon = pymysql.connect(host='127.0.0.1', user='******', passwd='******', db='csvimages', port=3306, charset='utf8') mycur =mycon.cursor() mycur.execute('select * from imgs where(`imgs`.`OriginalURL` like "%http%" and Title like "%ic%")') mydata = mycur.fetchall() mycon.commit() for data in mydata: url = data[2] filename = url.split('/')[-1] # print (filename) print("Downloading:", url) renum = 6 while os.path.exists(filename) == False and renum > 0: try: req = urllib2.Request(url) web = urllib2.urlopen(req, timeout=3) img = open(filename, 'wb') img.write(web.read()) img.close() break except IOError as e: print(e) except Exception as e: os.remove(filename) print(e) renum -= 1 finally: mycon.close() if __name__ == '__main__': for _ in range(3): D = CsvReaderImageTwo() D.action()
Downloading Result:
这个run 成功,后面one-person project就可以通过数据库查询、添加了,再通过PyQt加UI界面,一个实用小程序就OK了