scribe日誌系統安裝筆記

scribe日誌系統安裝筆記

目前web訪問日誌爲crontab定時清空，而且負載均衡後，訪問隨機分配到一臺服務器。程序出錯後需要分析日誌時，需要登錄幾臺機器查看，於是想用一臺主機作日誌主機，負責收集（按時間順序而不用sort?）、分析日誌。下面的是年前安裝scribe日誌系統的記錄。

一、軟件下載 

wget http://www.monkey.org/~provos/libevent-2.0.10-stable.tar.gz
wget http://nchc.dl.sourceforge.net/project/boost/boost/1.45.0/boost_1_45_0.tar.bz2
wget http://apache.etoak.com//incubator/thrift/0.5.0-incubating/thrift-0.5.0.tar.gz
wget https://download.github.com/facebook-scribe-2ee14d3.tar.gz
wget http://cloud.github.com/downloads/facebook/scribe/scribe-2.2.tar.gz

二、安裝步驟 

1～4系統已經安裝過，下面安裝略過，未安裝的可yum install libevent進行安裝。
1、gcc> 3.3.5 
gcc -v
which gcc
2、安裝ruby 
3、安裝python 
4、安裝libevent 
5、安裝 boost 
tar jxvf boost_1_45_0.tar.bz2
cd boost_1_45_0
./bootstrap.sh
./bjam -s HAVE_ICU=1 --prefix=/usr/local/boost --includedir=/usr/local/boost/include --libdir=/usr/local/boost/lib
./bjam install --prefix=/usr/local/boost
6、安裝facebook 服務 
安裝thrift
tar zxvf thrift-0.5.0.tar.gz
cd thrift-0.5.0
./configure --with-boost=/usr/local/boost --with-php-config=/usr/local/php5/bin/php-config
make

(出現如下錯誤，編輯文件lib/cpp/src/concurrency/ThreadManage.h修改爲
24 #include <boost/tr1/tr1/functional>後，重新make)

src/concurrency/ThreadManager.h:24:26: tr1/functional: No such file or
directory
In file included from src/concurrency/ThreadManager.cpp:20:
src/concurrency/ThreadManager.h:61: error: `std::tr1' has not been declared
src/concurrency/ThreadManager.h:61: error: ISO C++ forbids declaration
of `function' with no type
src/concurrency/ThreadManager.h:61: error: expected `;' before '<' token
src/concurrency/ThreadManager.h:182: error: `ExpireCallback' has not
been declared
src/concurrency/ThreadManager.h:182: error: ISO C++ forbids declaration
of `expireCallback' with no type
src/concurrency/ThreadManager.cpp:135: error: `ExpireCallback' has not
been declared
src/concurrency/ThreadManager.cpp:135: error: ISO C++ forbids
declaration of `expireCallback' with no type
src/concurrency/ThreadManager.cpp:145: error: `ExpireCallback' does not
name a type
src/concurrency/ThreadManager.cpp: In member function `virtual void
apache::thrift::concurrency::ThreadManager::Impl::removeExpiredTasks()':
src/concurrency/ThreadManager.cpp:533: error: `expireCallback_' was not
declared in this scope
src/concurrency/ThreadManager.cpp: At global scope:

make install
安裝fb303
cd contrib/fb303

(目錄下沒有Makefile.in，將thrift目錄下的Makefile.in至fb303目錄下)
./bootstrap.sh
./configure --with-boost=/usr/local/boost
make
make install
7、安裝scribe 
環境變量
export BOOST_ROOT=/usr/local/boost
export LD_LIBRARY_PATH=/usr/local/boost/lib::/usr/lob:/usr/local/lib
ldconfig -v
tar zxvf facebook-scribe-2ee14d3.tar.gz (scribe需要在github上下載最新版，scribe2.2發佈版與thrift0.5.0不兼容，Ying-Yi Liang搞定並更新了)

cd facebook-scribe-2ee14d3
./bootstrap.sh
./configure --with-boost=/usr/local/boost --prefix=/usr/local/scribe
make
make install

8、配置測試 
mkdir /usr/local/scribe/conf
cp examples/example1.conf /usr/local/scribe/conf
/usr/local/scribe/bin/scribed -c /usr/local/scribe/conf/example1.conf
/usr/local/scribe/bin/scribed -c /usr/local/scribe/conf/example1.conf &
nohup /usr/local/scribe/bin/scribed -c /usr/local/scribe/conf/example1.conf &
exit

測試程序：
mkdir  /tmp/scribetest
測試寫入：
echo  "hello world"| /usr/local/softs/facebook-scribe-2ee14d3/examples/scribe_cat test
查看結果：
cat  /tmp/scribetest/test/test_current

 

9、配置PHP接口

 1）生成PHP庫及配置目錄

cd /usr/local/scribe
/usr/local/bin/thrift -o . -I /usr/local/share/ --gen php /usr/local/share/fb303/if/fb303.thrift /usr/local/bin/thrift -o . -I /usr/local/share/ --gen php /usr/local/soft/facebook-scribe-2ee14d3/if/scribe.thrift cp /usr/local/soft/thrift-0.5.0/lib/php/src includes/ -r mkdir -p includes/packages/fb303 mkdir -p includes/packages/scribe mv gen-php/fb303/FacebookService.php gen-php/fb303/fb303_types.php includes/packages/fb303/ mv gen-php/scribe/scribe_types.php includes/packages/scribe/ mv gen-php/scribe/scribe.php includes/ rm -rf gen-php
2）編寫測試程序：
vi test.php <?php //參考http://www.ruturaj.net/scribe-php-logging $GLOBALS['THRIFT_ROOT'] = './includes'; include_once $GLOBALS['THRIFT_ROOT'] . '/scribe.php'; include_once $GLOBALS['THRIFT_ROOT'] . '/transport/TSocket.php'; include_once $GLOBALS['THRIFT_ROOT'] . '/transport/TFramedTransport.php'; include_once $GLOBALS['THRIFT_ROOT'] . '/protocol/TBinaryProtocol.php'; $msg1['category'] = 'keyword'; $msg1['message'] = "This is some message for the category/n"; $msg2['category'] = 'keyword'; $msg2['message'] = "Some other message for the category/n"; //$log_entry = new LogEntry( array('category'=>$category, 'category'=>$category) ) $entry1 = new LogEntry($msg1); $entry2 = new LogEntry($msg2); $messages = array($entry1, $entry2); $socket = new TSocket('localhost', 1463, true); $transport = new TFramedTransport($socket); //$protocol = new TBinaryProtocol($trans, $strictRead=false, $strictWrite=true) $protocol = new TBinaryProtocol($transport, false, false); //$scribe_client = new scribeClient($iprot=$protocol, $oprot=$protocol) $scribe_client = new scribeClient($protocol, $protocol); $transport->open(); $scribe_client->Log($messages); $transport->close(); ?>
3）執行程序：

/usr/local/php5/bin/php test.php
4）查看結果：
cat /tmp/scribetest/keyword/keyword_current

三、相關文章 
多個服務器日誌的排序合併： clfmerge
scribe的安裝與使用 
scribe-日誌收集器分析
使用Scribe來監控大型網絡系統
nginx-scribe-log
Scribe PHP logging
Pipe Apache (or any) Logs to Scribe

 

參考：

scribe日誌系統安裝筆記
Facebook scribe在Ubuntu下面安裝筆記