influxDB內存優化

查詢series cardinality命令

• InfluxDB官網說明：Why does series cardinality matter?

InfluxDB maintains an in-memory index of every series in the system. 
As the number of unique series grows, so does the RAM usage. 
High series cardinality can lead to the operating system killing the 
InfluxDB process with an out of memory (OOM) exception. 
See SHOW CARDINALITY to learn about the InfluxSQL commands for series cardinality.

// 翻譯
InfluxDB會在系統上爲每個series維護一個內存索引，
而隨着這些series的增加，RAM內存使用率也會增加。
如果series cardinality如果太高，就會導致操作系統觸發OOMKiller機制，
將Influxdb進程KILL掉. 
使用 SHOW CARDINALITY 命令可以查看到 series cardinality。

• SHOW CARDINALITY命令

-- show estimated cardinality of the series on current database
SHOW SERIES CARDINALITY
-- show estimated cardinality of the series on specified database
SHOW SERIES CARDINALITY ON mydb
-- show exact series cardinality
SHOW SERIES EXACT CARDINALITY
-- show series cardinality of the series on specified database
SHOW SERIES EXACT CARDINALITY ON mydb

• How can I remove series from the index?

To reduce series cardinality, series must be dropped from the index.
 DROP DATABASE, DROP MEASUREMENT, 
 and DROP SERIES will all remove series from the index 
 and reduce the overall series cardinality.

//大意
要減少或者刪除series cardinality, 需要刪除庫/表/series

將series由保存到內存改爲保存到TSI文件

修改配置

• 官網influxdb詳細配置說明

這裏我們需要修改的配置是index-version項,可以在influxdb.conf中[data]下修改，也可以通過環境變量INFLUXDB_DATA_INDEX_VERSION修改.

[data]
  # The type of shard index to use for new shards.  The default is an in-memory index that is
  # recreated at startup.  A value of "tsi1" will use a disk based index that supports higher
  # cardinality datasets.
  # 這個配置項默認值inmem,可以取消註釋修改爲tsi1，那麼後續的index將會保存在TSI文件中了.重啓生效.
  # index-version = "inmem"

• 其它關於內存的配置

重構TSI索引

• influx_inspect buildtsi命令
• Rebuild the TSI index

1. 停止InfluxDB服務
2. 刪除所有_series文件夾
   默認情況下,_series保存在/data/<dbName>/_series,檢查並刪除/data目錄下所有_series
3. 刪除所有index文件夾
   默認情況下,index文件夾在/data/<dbName/<rpName>/<shardID>/index
4. 使用influx_inspect重構TSI index

# 格式
influx_inspect buildtsi -datadir <data_dir> -waldir <wal_dir>
# 示例
influx_inspect buildtsi -datadir /data -waldir /wal

5. 啓動influxDB服務