ELK索引備份，遷移及恢復

      由於公司的elasticsearch集羣只是用了兩臺服務器。只要一臺服務器數據丟失elasticsearch將丟失一半數據。所以數據的備份恢復就相當重要。elasticsearch 快照和恢復模塊可以創建單個索引或者整個集羣的快照到遠程的倉庫實現數據的備份及恢復。

下面以備份恢復.kibana索引爲例

數據備份及恢復

1、修改elasticsearch配置文件

vim elasticsearch.yml

添加 path.repo: ["/mnt/bak"]

#設置倉庫路徑

備註如果你有多臺ealsticsearch服務器作爲集羣可以先關閉分片服務器只在master主節點服務器上更改。或者在創建庫的之前設置創建共享文件夾

2、創建倉庫

curl -XPUT  http://192.168.10.49:9200/_snapshot/my_backup -d '
{
    "type": "fs",
    "settings": {
        "location": "/mnt/bak",
"compress": true
    }
}
'

運行上面的命令後返回結果{"acknowledged":true}

說明倉庫創建成功。

compress 是否壓縮

max_snapshot_bytes_per_sec 製作快照的速度默認20mb/s

max_restore_bytes_per_sec 快照恢復的速度默認20mb/s

查看倉庫

curl -GET http://192.168.10.49:9200/_snapshot
{"my_backup":{"type":"fs","settings":{"compress":"true","location":"/mnt/bak"}}}

3、備份數據

curl -XPUT http://192.168.10.49:9200/_snapshot/my_backup/snapshot_20161207

執行上面的命令會快照ealsticsearch上所有的索引。

如果需要快照指定的索引

curl -XPUT http://192.168.10.49:9200/_snapshot/my_backup/snapshot_20161207 -d '
{
    "indices": ".kibana"
}
'

如果成功返回結果顯示{"accepted":true}

4、查看備份

curl -XGET http://192.168.10.49:9200/_snapshot/my_backup/snapshot_20161207

此命令是查看快照的情況返回的數據會比較多

{"snapshots":[{"snapshot":"snapshot_20161207","version_id":2030599,"version":"2.3.5","indices":["es-index-

.........中間N行數據..........

{"total":71,"failed":0,"successful":37}}]}

也可以使用下面的命令查看快照的狀態

curl -XGET http://192.168.10.49:9200/_snapshot/my_backup/snapshot_20161207/_status
{"snapshots":[{"snapshot":"snapshot_20161207","repository":"my_backup","state":"SUCCESS","shards_stats":{"initializing":0,"started":0,"finalizing":0,"done":1,"failed":0,"total":1},"stats":{"number_of_files":14,"processed_files":14,"total_size_in_bytes":143273,"processed_size_in_bytes":143273,"start_time_in_millis":1481060674078,"time_in_millis":50},"indices":{".kibana":{"shards_stats":{"initializing":0,"started":0,"finalizing":0,"done":1,"failed":0,"total":1},"stats":{"number_of_files":14,"processed_files":14,"total_size_in_bytes":143273,"processed_size_in_bytes":143273,"start_time_in_millis":1481060674078,"time_in_millis":50},"shards":{"0":{"stage":"DONE","stats":{"number_of_files":14,"processed_files":14,"total_size_in_bytes":143273,"processed_size_in_bytes":143273,"start_time_in_millis":1481060674078,"time_in_millis":50}}}}}}]}

5、取消備份

curl -XDELETE http://192.168.10.49:9200/_snapshot/my_backup/snapshot_20161207

6、恢復備份

curl -XPOST http://192.168.10.49:9200/_snapshot/my_backup/snapshot_20160812/_restore

如果你的是集羣而且在創建倉庫的時候沒有配置共享文件夾那會報下面的錯誤

{"error":"RepositoryException[[my_backup] failed to create repository]; nested: CreationException[Guice creation errors:\n\n1) Error injecting constructor, org.elasticsearch.repositories.RepositoryException: [my_backup] location [/mnt/bak] doesn't match any of the locations specified by path.repo because this setting is empty\n  at org.elasticsearch.repositories.fs.FsRepository.<init>(Unknown Source)\n  while locating org.elasticsearch.repositories.fs.FsRepository\n  while locating org.elasticsearch.repositories.Repository\n\n1 error]; nested: RepositoryException[[my_backup] location [/mnt/bak] doesn't match any of the locations specified by path.repo because this setting is empty]; ","status":500}

解決方法關閉支點服務器elasticsearch服務重新執行即可成功。

如果已經存在.kibana索引可以先關閉掉。

成功恢復數據後只有主節點服務器存在.kibana索引。我們希望所有的節點服務器都存在此索引時執行下面的命令

curl -XPUT 'http://192.168.10.49:9200/.kibanna/_settings' -d '
{
    "index" : {
        "number_of_replicas" : 1
    }
}'