Python網絡數據採集
本書採用簡潔強大的Python語言，介紹了網絡數據採集，併爲採集新式網絡中的各種數據類型提供了全面的指導。第 1部分重點介紹網絡數據採集的基本原理：如何用Python從網絡服務器請求信息，如何對服務器的響應進行基本處理，以及如何以自動化手段與網站進行交互。第 二部分介紹如何用網絡爬蟲測試網站，自動化處理，以及如何通過更多的方式接入網絡。
Web Scraping with Python 2nd - 2018.pdf
https://github.com/REMitchell/python-scraping 2000左右星
討論釘釘免費羣21745728 qq羣144081101 567351477
精通Python爬蟲框架Scrapy - 2018.pdf
Scrapy是使用Python開發的一個快速、高層次的屏幕抓取和Web抓取框架，用於抓Web站點並從頁面中提取結構化的數據。《精通Python爬蟲框架Scrapy》以Scrapy 1.0版本爲基礎，講解了Scrapy的基礎知識，以及如何使用Python和三方API提取、整理數據，以滿足自己的需求。
本書共11章，其內容涵蓋了Scrapy基礎知識，理解HTML和XPath，安裝Scrapy並爬取一個網站，使用爬蟲填充數據庫並輸出到移動應用中，爬蟲的強大功能，將爬蟲部署到Scrapinghub雲服務器，Scrapy的配置與管理，Scrapy編程，管道祕訣，理解Scrapy性能，使用Scrapyd與實時分析進行分佈式爬取。本書附錄還提供了各種軟件的安裝與故障排除等內容。 本書適合軟件開發人員、數據科學家，以及對自然語言處理和機器學習感興趣的人閱讀。
- 源碼 github星級 300左右
Learning Scrapy -2016.pdf 另有中文電子版本 因爲版權已經在CSDN等網站下架，可以在qq羣144081101等找到。
精通Scrapy網絡爬蟲
本書深入系統地介紹了Python流行框架Scrapy的相關技術及使用技巧。全書共14章，從邏輯上可分爲基礎篇和高級篇兩部分，基礎篇重點介紹Scrapy的核心元素，如spider、selector、item、link等；高級篇講解爬蟲的高級話題，如登錄認證、文件下載、執行JavaScript、動態網頁爬取、使用HTTP代理、分佈式爬蟲的編寫等，並配合項目案例講解，包括供練習使用的網站，以及知乎、豆瓣、360爬蟲案例等。 本書案例豐富，注重實踐，代碼註釋詳盡，適合有一定Python語言基礎，想學習編寫複雜網絡爬蟲的讀者使用。
python3爬蟲基礎
https://github.com/MorvanZhou/easy-scraping-tutorial 200 左右星
First web scraper
教程：https://first-web-scraper.readthedocs.io/en/latest/
https://github.com/ireapps/first-web-scraper/blob/master/docs/index.rst 200 左右星
Practical Web Scraping for Data Science -Best Practices and Examples with Python - 2018.pdf
https://github.com/Apress/practical-web-scraping-for-data-science 星級 低於100
This book provides a complete and modern guide to web scraping, using Python as the programming language, without glossing over important details or best practices. Written with a data science audience in mind, the book explores both scraping and the larger context of web technologies in which it operates, to ensure full understanding. The authors recommend web scraping as a powerful tool for any data scientist’s arsenal, as many data science projects start by obtaining an appropriate data set.
Starting with a brief overview on scraping and real-life use cases, the authors explore the core concepts of HTTP, HTML, and CSS to provide a solid foundation. Along with a quick Python primer, they cover Selenium for JavaScript-heavy sites, and web crawling in detail. The book finishes with a recap of best practices and a collection of examples that bring together everything you've learned and illustrate various data science use cases.
用Python寫網絡爬蟲 第2版
《用Python寫網絡爬蟲（第 2版》講解了如何使用Python來編寫網絡爬蟲程序，內容包括網絡爬蟲簡介，從頁面中抓取數據的3種方法，提取緩存中的數據，使用多個線程和進程進行併發抓取，抓取動態頁面中的內容，與表單進行交互，處理頁面中的驗證碼問題，以及使用Scarpy和Portia進行數據抓取，並在最後介紹了使用本書講解的數據抓取技術對幾個真實的網站進行抓取的實例，旨在幫助讀者活學活用書中介紹的技術。
《用Python寫網絡爬蟲（第 2版》適合有一定Python編程經驗而且對爬蟲技術感興趣的讀者閱讀。
Python Web Scraping 2nd Edition - 2017.pdf
第一版中文 用Python寫網絡爬蟲.pdf
https://github.com/kjam/wswp < 100星
Python Web Scraping Cookbook - 2018.pdf
Python Web Scraping Cookbook is a solution-focused book that will teach you techniques to develop high-performance Scrapers, and deal with cookies, hidden form fields, Ajax-based sites and proxies. You'll explore a number of real-world scenarios where every part of the development or product life cycle will be fully covered. You will not only develop the skills to design reliable, high-performing data flows, but also deploy your codebase to Amazon Web Services (AWS). If you are involved in software engineering, product development, or data mining or in building data-driven products, you will find this book useful as each recipe has a clear purpose and objective.
Right from extracting data from websites to writing a sophisticated web crawler, the book's independent recipes will be extremely helpful while on the job. This book covers Python libraries, requests, and BeautifulSoup. You will learn about crawling, web spidering, working with AJAX websites, and paginated items. You will also understand to tackle problems such as 403 errors, working with proxy, scraping images, and LXML.
By the end of this book, you will be able to scrape websites more efficiently and deploy and operate your scraper in the cloud.
https://github.com/PacktPublishing/Python-Web-Scraping-Cookbook < 100星
Website Scraping with Python - 2018.pdf
仔細檢查網站抓取和數據處理：以適合進一步分析的格式從網站提取數據的技術。您將查看要使用的工具，並比較它們的功能和效率。本書簡明扼要專注於BeautifulSoup4和Scrapy，突出了常見問題，並提出了讀者可以自行實施的解決方案。
您將看到如何單獨或一起使用BeautifulSoup4和Scrapy以獲得所需的結果。由於許多站點都使用JavaScript，因此您還將使用Selenium和瀏覽器模擬器來呈現這些站點。
在本書的最後，您將擁有一個完整的抓取應用程序來使用和重寫以滿足您的需求。
https://github.com/Apress/website-scraping-w-python
Social Media Data Mining and Analytics - 2018.pdf
Harness the power of social media to predict customer behaviorand improve sales
Social media is the biggest source of Big Data. Because of this,90% of Fortune 500 companies are investing in Big Data initiativesthat will help them predict consumer behavior to produce bettersales results. Written by Dr. Gabor Szabo, a Senior Data Scientistat Twitter, and Dr. Oscar Boykin, a Software Engineer at Twitter,Social Media Data Mining and Analytics shows analysts how touse sophisticated techniques to mine social media data, obtainingthe information they need to generate amazing results for theirbusinesses. Social Media Data Mining and Analytics isn’t just anotherbook on the business case for social media. Rather, this bookprovides hands-on examples for applying state-of-the-art tools andtechnologies to mine social media – examples include Twitter,Facebook, Pinterest, Wikipedia, Reddit, Flickr, Web hyperlinks, andother rich data sources. In it, you will learn:
The four key characteristics of online services-users, socialnetworks, actions, and content The full data discovery lifecycle-data extraction, storage,analysis, and visualization How to work with code and extract data to create solutions How to use Big Data to make accurate customer predictions
Szabo and Boykin wrote this book to provide businesses with thecompetitive advantage they need to harness the rich data that isavailable from social media platforms.
參考資料
- 討論qq羣144081101 591302926 567351477 釘釘羣21745728
- 本文最新版本地址
- 本文涉及的python測試開發庫 謝謝點贊！
- 本文相關海量書籍下載
- 2018最佳人工智能機器學習工具書及下載(持續更新)
- https://github.com/lorien/awesome-web-scraping/blob/master/python.md
- 最好用的Python爬蟲推薦 https://www.jianshu.com/p/7da43c16dd87
- https://www.zhihu.com/question/41277528