大數據爬蟲基礎（三）Scrapy在ubuntu 16.04下的安裝

Scrapy ubuntu下安裝

系統：ubuntu 16.04 no gui

依賴包及依賴包的依賴包：
下列的安裝步驟假定您已經安裝好下列程序:
http://scrapy.org/
Python 2.7
Python Package: pip and setuptools. 現在 pip 依賴 setuptools ，如果未安裝，則會自動安裝 setuptools 。
lxml. 大多數Linux發行版自帶了lxml。如果缺失，請查看http://lxml.de/installation.html
OpenSSL. 除了Windows(請查看 平臺安裝指南)之外的系統都已經提供。
您可以使用pip來安裝Scrapy(推薦使用pip來安裝Python package).


使用pip安裝:
小寫scrapy不是大寫，官網是小寫
http://scrapy-chs.readthedocs.org/zh_CN/latest/intro/install.html
pip install scrapy


1、pip,easy_install
Ubuntu下安裝pip的方法
 http://www.2cto.com/os/201305/213725.html
安裝pip的方法：
Install pip and virtualenv for Ubuntu 10.10 Maverick and newer
 
$ sudo apt-get install python-pip python-dev build-essential 
$ sudo pip install --upgrade pip 
$ sudo pip install --upgrade virtualenv 
For older versions of Ubuntu
 
Install Easy Install
$ sudo apt-get install python-setuptools python-dev build-essential 
Install pip
$ sudo easy_install pip 
Install virtualenv
$ sudo pip install --upgrade virtualenv 
sudo apt-get install python-setuptools python-dev build-essential




2、lxml
先安裝依賴包，否則裝不上，報錯：x86_64-linux-gnu-gcc error
裝：apt-get install -y libxml2-dev libxslt1-dev zlib1g-dev python3-pip


或者裝：apt-get install build-essential autoconf libtool pkg-config python-opengl python-imaging python-pyrex python-pyside.qtopengl idle-python2.7 qt4-dev-tools qt4-designer libqtgui4 libqtcore4 libqt4-xml libqt4-test libqt4-script libqt4-network libqt4-dbus python-qt4 python-qt4-gl libgle3 python-dev


success後
再pip install lxml
success


3、cryptography及其依賴包
直接pip install scrapy會報 cryptography和cffi的錯誤，安裝以下依賴包：
https://cryptography.io/en/latest/installation/#building-cryptography-on-linux
apt-get install build-essential libssl-dev libffi-dev python-dev
success


4、Scrapy
pip install scrapy
success




5、可選包：
pip install pymongo
pip install pillow
pip install pycrypto


6、報錯參考
http://stackoverflow.com/questions/22073516/failed-to-install-python-cryptography-package-with-pip-and-setup-Python
http://stackoverflow.com/questions/27130286/error-command-x86-64-linux-gnu-gcc-failed-with-exit-status-1-in-virtualenv
http://www.cnblogs.com/lyroge/archive/2013/02/22/2922515.html