Centos6.5x64 安裝scrapy1.0.4成功

最近在自學爬蟲框架scrapy，看了看官網文檔，迫不及待動手安裝。結果錯誤百出，慘不忍睹。網上搜了一篇文章，（原文鏈接 http://www.tuicool.com/articles/URNVV3E）。

首先，介紹下環境：

1、Centos6.5 x64 安裝在VMware虛擬機中，因爲後期需要移植，遂選擇了兼容vm10.0。

# yum -y update

升級下系統

2、Python2.7.9。系統自帶爲2.6.6。

cd ~/Download

管網下載源碼：

wget --no-check-certificate https://www.python.org/ftp/python/2.7.9/Python-2.7.9.tar.xz

tar xvf Python-2.7.9.tar.xz

cd Python-2.7.9

./configure --with-ensurepip=install   #其餘保持默認

make

sudo make install  # 如果python2.7.9做副版本的話使用make altinstall

[Mikky@localhost Python-2.7.9]$ python
Python 2.7.9 (default, Feb  1 2016, 21:30:54)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

升級了下libffi-devel因爲踩它坑太多，所以直接又更新了下，此步選做：

sudo yum install update libffi-devel

至此環境準備妥當，下面就是踩坑環節o(╯□╰)o

安裝失敗方法：

sudo /usr/local/bin/pip install scrapy

錯誤很多，參考文首的鏈接依然無法解決。坑就不貼出來了，慢慢踩吧。

那麼多坑的主要原因在於pip安裝軟件的依賴庫問題，尤其是：cryptography。

pip默認安裝最新的而不是最穩定的版本，安裝的cryptography爲最新版（1.2.2    我的安裝時間爲2016年2月初），因此：

sudo /usr/local/bin/pip install cryptography==0.9

........無視warn

Successfully installed cryptography idna pyasn1 six enum34 ipaddress cffi pycparser
Cleaning up...

到這大功告成一大半了，接下來請主角出場：

sudo /usr/local/bin/pip install scrapy
.........

Successfully installed scrapy pyOpenSSL queuelib service-identity lxml w3lib cssselect Twisted pyasn1-modules characteristic zope.interface
Cleaning up...

[Mikky@localhost ~]$ scrapy version
/usr/local/lib/python2.7/site-packages/cffi/model.py:526: UserWarning: 'point_conversion_form_t' has no values explicitly defined; next version will refuse to guess which integer type it is meant to be (unsigned/signed, int/long)
  % self._get_c_name())
Scrapy 1.0.4
繼續無視警告，已經被這些警告整瘋了。

$  scrapy startproject tutorial
/usr/local/lib/python2.7/site-packages/cffi/model.py:526: UserWarning: 'point_conversion_form_t' has no values explicitly defined; next version will refuse to guess which integer type it is meant to be (unsigned/signed, int/long)
  % self._get_c_name())
New Scrapy project 'tutorial' created in:
    /home/Mikky/scrapy/tutorial

You can start your first spider with:
    cd tutorial
    scrapy genspider example example.com
[Mikky@localhost scrapy]$ ll
total 4
drwxrwxr-x. 3 Mikky Mikky 4096 Feb  3 22:27 tutorial

安裝完成，關於這個warning，參見https://bitbucket.org/cffi/cffi/issues/234/cffi-131-userwarning-about  。