從安裝supervisor到管理pyspider進程

http://www.shsay.com

 

原文鏈接http://blog.csdn.net/xyang81/article/details/51555473

pyspider配置https://www.cnblogs.com/microman/p/6138082.html

安裝

pip install supervisor

 

運行echo_supervisord_conf程序生成supervisor的初始化配置文件

echo_supervisord_conf > /etc/supervisord.conf

 

mkdir supervisor_conf

修改/etc/supervisord.conf中的include參數，將/etc/supervisor_conf目錄添加到include中

vim /etc/supervisord.conf

前面的分號表示註釋，記得去掉

 

[include]
files = /etc/supervisor_conf/*.ini

 

cd /etc/supervisord.conf/

vim pyspider.ini

內容放到文章最後

 

cd /home/c_pyspider/

mkdir logs

cd logs

vim webui.log

vim scheduler.log

vim processor.log

vim worker.log

vim fetcher.log

vim phantomjs.log

 

啓動服務

supervisord -c /etc/supervisord.conf

 

注意：在執行第一條命令出現以下提示信息時：

Error: Another program is already listening on a port that one of our HTTP servers is configured to use.  
Shut this program down first before starting supervisord.
For help, use /usr/bin/supervisord –h

是因爲有一個使用supervisor配置的應用程序正在運行，需要在supervisorctl交互終端執行 shutdown命令終止它，或者執行reload 來重新啓動配置中的所有程序，或者在xhell命令行直接執行

 

find / -name supervisor.sock

unlink /tmp/supervisor.sock

常用命令：                                                                                                                                 

 

 

service supervisord stop

service supervisord start

 

二、更新新的配置到supervisord    

[vb] view plain copy

1. supervisorctl update  

三、重新啓動配置中的所有程序

[vb] view plain copy

1. supervisorctl reload  

四、啓動某個進程(program_name=你配置中寫的程序名稱)

[vb] view plain copy

1. supervisorctl start program_name  

五、查看正在守候的進程

[vb] view plain copy

1. supervisorctl  

六、停止某一進程 (program_name=你配置中寫的程序名稱)

[vb] view plain copy

1. pervisorctl stop program_name  

七、重啓某一進程 (program_name=你配置中寫的程序名稱)

[vb] view plain copy

1. supervisorctl restart program_name  

八、停止全部進程

[vb] view plain copy

1. supervisorctl stop all  

注意：顯示用stop停止掉的進程，用reload或者update都不會自動重啓。

 

 

 

其它：開啓web界面和配置開機啓動

 

[group:pyspider]
program=pyspider-webui,pyspider-scheduler,pyspider-processor,pyspider-result_worker,pyspider-fetcher,pyspider-phantomjs
priority=999

[program:pyspider-webui]
directory=/home/c_pyspider
command=pyspider -c config.json webui
autostart=true
autorestart=true
priority=905
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/webui.log


[program:pyspider-scheduler]
directory=/home/c_pyspider
command=pyspider -c config.json scheduler
autostart=true
autorestart=true
priority=900
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/scheduler.log

[program:pyspider-processor]
command=pyspider -c config.json  processor
directory=/home/c_pyspider
autostart=true
autorestart=true
priority=903
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/processor.log

[program:pyspider-result_worker]
command=pyspider -c config.json result_worker
directory=/home/c_pyspider
autostart=true
autorestart=true
priority=904
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/worker.log

[program:pyspider-fetcher]
command=pyspider -c config.json  --phantomjs-proxy="localhost:25555" fetcher
directory=/home/c_pyspider
autostart=true
autorestart=true
priority=902
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/fetcher.log

[program:pyspider-phantomjs]
command=pyspider -c config.json phantomjs
directory=/home/c_pyspider
autostart=true
autorestart=true
stopasgroup=true
stopsignal=QUIT
;上面兩條針對supervisor停止phantomjs進程
priority=901
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /home/c_pyspider/logs/phantomjs.log