"本文主要讲解fluent-bit和fluentd安装配置和使用介绍"
1、环境说明
两台服务器（注：Fluent-bit只支持centos 7以上版本，Fluentd可以支持centos 6版本），host1写日志到本地，然后通过Fluent-bit支持的forward到Fluentd，Fluentd将日志集中写入host2本地存储归档。
host1，IP:12.18.7.42，CentOS 7.6
host2，IP:12.18.7.41，CentOS 7.6
2、fluent-bit安装过程
2.1、安装编译运行环境
yum install cmake gcc gcc-g++ -y
2.2、下载fluentbit
wget https://fluentbit.io/releases/1.0/fluent-bit-1.0.4.tar.gz
2.3、安装
tar -zxf fluent-bit-1.0.4.tar.gz
cd fluent-bit-1.0.4/build
cmake..
make
make install
2.4、 执行
fluent-bit -c /usr/local/etc/fluent-bit/fluent-bit.conf
Fluent Bit v1.0.4
Copyright (C) Treasure Data
[2020/01/19 11:34:39] [Warning] [config] I cannot open /usr/local/etc/fluent-bit/plugins.conf file
[2020/01/19 11:34:39] [ info] [storage] initializing...
[2020/01/19 11:34:39] [ info] [storage] in-memory
[2020/01/19 11:34:39] [ info] [storage] normal synchronization mode, checksum disabled
[2020/01/19 11:34:39] [ info] [engine] started (pid=3270)
[0] cpu.local: [1579404880.000151741, {"cpu_p"=>0.000000, "user_p"=>0.000000, "system_p"=>0.000000, "cpu0.p_cpu"=>0.000000, "cpu0.p_user"=>0.000000, "cpu0.p_syst
3、fluentd安装
Fluentd是用ruby语言写的日志收集工具。安装ruby默认是2.0版本的，需要通过yum安装ruby环境。
3.1、安装ruby环境
[root@host2 src]# yum install ruby
3.2、修改成阿里源，能够加快下载速度，添加ruby仓库
[root@host2 src]# gem sources -a http://mirrors.aliyun.com/rubygems/
3.3、安装RVM
RVM（Ruby Version Manager ）是一款RVM的命令行工具，可以使用RVM轻松安装，管理Ruby版本。RVM包含了Ruby的版本管理和Gem库管理(gemset)
[root@k8s-master src]# gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BD4
[root@k8s-master src]# curl -sSL https://get.rvm.io | bash -s stable
3.4、生效配置
[root@host2 src]# source /etc/profile.d/rvm.sh
3.5、查看版本
[root@host2 src]# rvm -v
rvm 1.29.9 (latest) by Michal Papis, Piotr Kuczynski, Wayne E. Seguin [https://rvm.io]
3.6、安装Ruby
[root@host2 src]# rvm install 2.5
3.7、安装编译环境和软件包
[root@host2 src]# yum install gcc-c++ make automake autoconf libtool openssl-devel jemalloc-devel gmp-devel -y
3.8、安装Fluentd（当然你也可以安装必要的的插件，比如：gem install fluent-plugin-elasticsearch）
[root@host2 src]# gem install fluentd --no-document
3.9、初始化Fluentd
[root@host2 src]# fluentd --setup /etc/fluentd
3.10、启动
[root@host2 src]# fluentd -c /etc/fluentd/fluent.conf
[root@k8s-master src]# fluentd -c 13_fluentd_conf
2020-01-19 10:43:18 +0800 [info]: parsing config file is succeeded path="13_fluentd_conf"
2020-01-19 10:43:18 +0800 [info]: gem 'fluentd' version '1.8.1'
2020-01-19 10:43:18 +0800 [info]: Oj is not installed, and failing back to Yajl for json parser
2020-01-19 10:43:18 +0800 [info]: using configuration file: <ROOT>
<source>
@type forward
port 24222
</source>
<source>
@type http
port 9880
</source>
<match nginx*>
@type stdout
</match>
</ROOT>
2020-01-19 10:43:18 +0800 [info]: starting fluentd-1.8.1 pid=14389 ruby="2.5.5"
2020-01-19 10:43:18 +0800 [info]: spawn command to main: cmdline=["/usr/local/rvm/rubies/ruby-2.5.5/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/local/rvm/gems/ruby-2.5.5/bin/fluentd", "-c", "13_fluentd_conf", "--under-supervisor"]
2020-01-19 10:43:19 +0800 [info]: adding match pattern="nginx*" type="stdout"
2020-01-19 10:43:19 +0800 [info]: adding match pattern="nginx*" type="file"
2020-01-19 10:43:19 +0800 [info]: adding source type="forward"
2020-01-19 10:43:19 +0800 [info]: adding source type="http"
2020-01-19 10:43:19 +0800 [info]: #0 Oj is not installed, and failing back to Yajl for json parser
2020-01-19 10:43:19 +0800 [info]: #0 starting fluentd worker pid=14393 ppid=14389 worker=0
2020-01-19 10:43:19 +0800 [info]: #0 listening port port=24222 bind="0.0.0.0"
2020-01-19 10:43:19 +0800 [info]: #0 fluentd worker is now running worker=0
4、使用样例
4.1、启动fluentd服务端，命令如下：
fluentd -c fluentd_conf
fluent_conf信息如下：
<source>
@type forward
port 24222
</source>
# http://this.host:9880/myapp.access?json={"event":"data"}
<source>
@type http
port 9880
</source>
<match nginx*> #匹配有tag为mem的类型
@type stdout #匹配成功直接标准输出
</match>
4.2、启动fluent-bit客户端
fluent-bit -c fluent-bit.conf
fluent-bit.conf配置文件信息如下：
[SERVICE]
Flush 1
Daemon OFF
Log_Level debug
[INPUT]
Name tail
Path /home/logs/ng.log
Db /tmp/ng.db
Db.sync Full
Tag nginxlog8
[OUTPUT]
Name forward
Match *
Host 12.18.7.41
Port 24222
4.3、日志输出测试
fluent-bit客户端执行
echo qquuuuuu>ng.log
flunetd服务端输出：
2020-01-19 13:52:38.945508352 +0800 nginxlog8: {"log":"qquuuuuu"}
5、总结
本文主要讲解了fluentd和fluent-bit原生安装和入门使用的过程，上一篇介绍了为什么用？Kubernetes集群环境下fluentd日志收集方案介绍 后续会介绍文件按照规范存储日志文件以及镜像DaemonSet方式运行，持续关注。
推荐阅读
Kubernetes中如何使用ClusterDNS进行服务发现？
原创不易，随手关注或者”在看“，诚挚感谢！