(一)背景介紹
zabbix是一個大型的分佈式的監控系統,監控的範圍比較廣,是目前比較流行的監控系統,但是由於自身的原因,歷史數據不能持久保存,如果數據庫的數據大於100G左右查詢或其他的速度會非常的慢,會觸發很多問題,一般的zabbix歷史數據會不超過一個月(按實際獲得的數據比例計算),我們一般保存七天。一般爲了業務的需要,往往會需要很長的歷史數據來進行查看和排查問題,這就需要使zabbix的歷史數據進行長久保存(不能存數據庫,而可以存ES存儲)。
(二)環境
zabbix:zabbix4.0.1(安裝部署省略)
ES:5.5.2 (安裝部署省略)
(三)具體的配置
3.1、首先修改zabbix_server.conf文件,啓用歷史數據的配置,具體如下:
$grep '^[a-Z]' /etc/zabbix/zabbix_server.conf
LogFile=/tmp/zabbix_server.log
LogFileSize=0
PidFile=/var/run/zabbix/zabbix_server.pid
SocketDir=/var/run/zabbix
DBHost=localhost
DBName=zabbix
DBUser=zabbix
DBPassword=zabbix
HistoryStorageURL=http://X.X.X.X:9200
HistoryStorageTypes=uint,db1,str,log,text
HistoryStorageDateIndex=1
SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
Timeout=4
AlertScriptsPath=/usr/lib/zabbix/alertscripts
ExternalScripts=/usr
########################下邊是具體的解釋####################################
### Option: HistoryStorageURL
# History storage HTTP[S] URL.
#
# Mandatory: no
# Default:
# HistoryStorageURL=
#HistoryStorageURL="-uelastic:FatPPdaiCom http://10.114.16.77:9200"
HistoryStorageURL=http://10.114.23.139:9200
### Option: HistoryStorageTypes
# Comma separated list of value types to be sent to the history storage.
#
# Mandatory: no
# Default:
# HistoryStorageTypes=uint,db1,str,log,text
HistoryStorageTypes=uint,db1,str,log,text
### Option: HistoryStorageDateIndex
# Enable preprocessing of history values in history storage to store values in different indices based on date.
# 0 - disable
# 1 - enable
#
# Mandatory: no
# Default:
HistoryStorageDateIndex=1
備註:下邊是ES所支持存儲的數據類型
3.2、修改zabbix前端的配置文件,添加global $DB,$HISTORY;
$vim /etc/zabbix/web/zabbix.conf.php
<?php
// Zabbix GUI configuration file.
global $DB,$HISTORY;
#global $DB;
$DB['TYPE'] = 'MYSQL';
$DB['SERVER'] = 'localhost';
$DB['PORT'] = '0';
$DB['DATABASE'] = 'zabbix';
$DB['USER'] = 'zabbix';
$DB['PASSWORD'] = 'zabbix';
// Schema name. Used for IBM DB2 and PostgreSQL.
$DB['SCHEMA'] = '';
$ZBX_SERVER = '10.114.23.230';
$ZBX_SERVER_PORT = '10051';
$ZBX_SERVER_NAME = 'zabbix';
$IMAGE_FORMAT_DEFAULT = IMAGE_FORMAT_PNG;
// Elasticsearch url (can be string if same url is used for all types).
$HISTORY['url'] = 'http://X.X.X.X:9200';
#$HISTORY['url'] = '-uelastic:FatPPdaiCom http://X.X.X.X:9200';
// Value types stored in Elasticsearch.
$HISTORY['types'] = ['uint', 'text', 'log', 'str', 'db1'];
//--------------------------------------------------------------------------------------------------------------------------
//如果不同類型使用不同的 ES 集羣,可以按如下進行配置
#$HISTORY['url'] = [
# 'uint' => 'http://10.114.16.77:9200 ',
# 'text' => 'http://10.114.16.77:9200 '
#];
#$HISTORY['types'] = ['uint', 'text'];
注意的要點:
1、HISTORY['url']和HISTORY['types']被更新。
2、定義zabbix GUI全局數據源路徑文件 global $DB, $HISTORY;(告訴zabbix先從數據庫裏讀取,沒有的話取HISTORY讀取)
3.3、在ES上依次創建五個索引(不創建也可以,會自動創建的)
$curl -XPUT http://X.X.X.X:9200/unit \
-H 'content-type:application/json' \
-d '{
"settings" : {
"index" : {
"number_of_replicas" : 1,
"number_of_shards" : 5
}
},
"mappings" : {
"values" : {
"properties" : {
"itemid" : {
"type" : "long"
},
"clock" : {
"format" : "epoch_second",
"type" : "date"
},
"value" : {
"type" : "long"
}
}
}
}
}'
$curl -XPUT http://X.X.X.X:9200/db1 \
-H 'content-type:application/json' \
-d '{
"settings" : {
"index" : {
"number_of_replicas" : 1,
"number_of_shards" : 5
}
},
"mappings" : {
"values" : {
"properties" : {
"itemid" : {
"type" : "long"
},
"clock" : {
"format" : "epoch_second",
"type" : "date"
},
"value" : {
"type" : "double"
}
}
}
}
}'
$curl -XPUT http://X.X.X.X:9200/log \
-H 'content-type:application/json' \
-d '{
"settings" : {
"index" : {
"number_of_replicas" : 1,
"number_of_shards" : 5
}
},
"mappings" : {
"values" : {
"properties" : {
"itemid" : {
"type" : "long"
},
"clock" : {
"format" : "epoch_second",
"type" : "date"
},
"value" : {
"fields" : {
"analyzed" : {
"index" : true,
"type" : "text",
"analyzer" : "standard"
}
},
"index" : false,
"type" : "text"
}
}
}
}
}'
$curl -XPUT http://X.X.X.X:9200/text \
-H 'content-type:application/json' \
-d '{
"settings" : {
"index" : {
"number_of_replicas" : 1,
"number_of_shards" : 5
}
},
"mappings" : {
"values" : {
"properties" : {
"itemid" : {
"type" : "long"
},
"clock" : {
"format" : "epoch_second",
"type" : "date"
},
"value" : {
"fields" : {
"analyzed" : {
"index" : true,
"type" : "text",
"analyzer" : "standard"
}
},
"index" : false,
"type" : "text"
}
}
}
}
}'
$curl -XPUT http://X.X.X.X:9200/str \
-H 'content-type:application/json' \
-d '{
"settings" : {
"index" : {
"number_of_replicas" : 1,
"number_of_shards" : 5
}
},
"mappings" : {
"values" : {
"properties" : {
"itemid" : {
"type" : "long"
},
"clock" : {
"format" : "epoch_second",
"type" : "date"
},
"value" : {
"fields" : {
"analyzed" : {
"index" : true,
"type" : "text",
"analyzer" : "standard"
}
},
"index" : false,
"type" : "text"
}
}
}
}
}'
3.4、在ES上查看是否生成:
$curl http://X.X.X.X:9200/_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open str ZTkdqhpFR9ijzTfKvtsMrQ 5 1 0 0 2.2kb 1.1kb
green open text TjY6OEGySJ2AGc1rpfOEhA 5 1 0 0 2.2kb 1.1kb
green open log qG-R2UhiQhypP9stSSodkw 5 1 0 0 2.2kb 1.1kb
green open db1 av_V5XbiSDyDdZXW0rO6aw 5 1 0 0 2.2kb 1.1kb
green open .kibana JZBiyypQSRuMxRnHxOLP1Q 1 1 1 0 8kb 4kb
green open unit 2bxxvaMTTFKqTq0EDX5EjA 5 1 0 0 2.2kb 1.1kb
3.5、在kibana上創建相應的索引並查看數據。
就可以看到歷史數據會存儲到ES中去
(四)、查看是否通過db訪問還是es,修改該文件(/etc/zabbix/web/zabbix.conf.php)的全局變量就可以知道訪問的是mysql還是ES了。
具體可以參考官方文檔(https://www.zabbix.com/documentation/4.0/manual/appendix/install/elastic_search_setup)