ELK5.6.1集羣升級到6.6.0集羣更新文檔(完整版)
環境介紹:
elasticsearch-5.6.1 --- elasticsearch-6.6.0
kibana-5.6.1 --- kibana-6.6.0
Logstash-5.6.1 --- Logstash-6.6.0
IP | 角色 |
10.0.9.24 | Elasticsearch,Kibana |
10.0.9.25 | Elasticsearch(master) |
10.0.8.50 | Logstash |
一、Elasticsearch升級
1.備份所有索引
①修改配置文件,增加備份倉庫設置
vim elasticsearch.yml
path.repo: ["/elk/data/bak"] #上一級文件夾必須存在,這個路徑必須是一個網盤,如NFS
②創建倉庫名
curl -XPUT http://10.0.9.25:9200/_snapshot/my_fs_backup -H 'Content-Type: application/json' -d'{"type": "fs","settings": {"location": "/elk/data/bak/Backup20190531.bk","compress": true}}'
③向倉庫備份index
curl -XPUT http://10.0.9.25:9200/_snapshot/my_fs_backup/snapshot_1?wait_for_completion=true
④查看備份狀態,信息
curl -XGET http://10.0.9.25:9200/_snapshot/my_backup/snapshot_1 curl -XGET http://10.0.9.25:9200/_snapshot/my_backup/_all curl -XGET http://10.0.9.25:9200/_snapshot/my_backup/snapshot_1/status
2.停止碎片分配
curl -XPUT '10.0.9.24:9200/_cluster/settings?pretty' -H 'Content-Type: application/json' -d' { "persistent": { "cluster.routing.allocation.enable": "none" } }'
3.刷新同步狀態(可選)
curl -XPOST '10.0.9.24:9200/_flush/synced?pretty'
4.關閉所有節點
①ps -ef|grep elastic
②kill -9 pid
5.升級軟件版本
①tar xvf elasticsearch-6.6.0.tar.gz
②mv elasticsearch-6.6.0 /elk/
6.配置文件修改,重新指向原有data目錄和logs目錄
①cd /elk/elasticsearch-6.6.0
②vim config/elasticsearch.yml
cluster.name: myapp node.name: elk_node1 path.data: /elk/data bootstrap.memory_lock: true bootstrap.system_call_filter: false network.host: 10.0.9.25 http.port: 9200 discovery.zen.ping.unicast.hosts: ["10.0.9.24", "10.0.9.25"] discovery.zen.minimum_master_nodes: 1 #如果有3個節點,這裏配置爲2 gateway.recover_after_nodes: 1 gateway.expected_nodes: 2 gateway.recover_after_time: 4m action.auto_create_index: true cluster.routing.allocation.same_shard.host: true thread_pool.bulk.queue_size: 1000 node.data: true path.repo: ["/elk/data/bak"]
7.升級x-pack索引
①獲取x-pack索引狀態
curl -XGET 10.0.9.24:9200/_xpack/migration/assistance
②升級所有索引
curl -XPOST 10.0.9.24:9200/_xpack/migration/upgrade/.security curl -XPOST 10.0.9.24:9200/_xpack/migration/upgrade/.watches curl -XPOST 10.0.9.24:9200/_xpack/migration/upgrade/.triggered_watches
8.啓動服務,查看健康狀態
①啓動elasticsearch
./bin/elasticsearch -d
curl -XGET '10.0.9.24:9200/_cat/health?pretty' curl -XGET '10.0.9.24:9200/_cat/nodes?pretty'
9.恢復碎片分配
curl -XPUT '10.0.9.24:9200/_cluster/settings?pretty' -H 'Content-Type: application/json' -d' { "persistent": { "cluster.routing.allocation.enable": "true" } }'
二、Kibana升級
1 . 解壓和複製
①tar xvf kibana-6.6.0-linux-x86_64.tar.gz
②mv kibana-6.6.0-linux-x86_64 /elk
2 . 配置文件修改
①cd /elk/kibana-6.6.0-linux-x86_64
②vim config/kibana.yml
server.port: 7601 server.host: "10.0.9.24" elasticsearch.hosts: "http://10.0.9.24:9200" #elasticsearch.username: "elastic" #這裏必須和X-PACK破解結合 #elasticsearch.password: "changeme" pid.file: /elk/kibana-6.6.0-linux-x86_64/kibana.pid logging.dest: stdout
3 . 重建.kibana索引
①設置.kibana的index空間爲只讀(block.sh)
curl -XPUT '10.0.9.24:9200/.kibana/_settings?pretty' -H 'Content-Type: application/json' -d' { "index.blocks.write": true }'
②創建新的index空間.kibana-6(new_space.sh)
curl -XPUT '10.0.9.24::9200/.kibana-6?pretty' -H 'Content-Type: application/json' -d' { "settings" : { "number_of_shards" : 1, "index.mapper.dynamic": false }, "mappings" : { "doc": { "properties": { "type": { "type": "keyword" }, "updated_at": { "type": "date" }, "config": { "properties": { "buildNum": { "type": "keyword" } } }, "index-pattern": { "properties": { "fieldFormatMap": { "type": "text" }, "fields": { "type": "text" }, "intervalName": { "type": "keyword" }, "notExpandable": { "type": "boolean" }, "sourceFilters": { "type": "text" }, "timeFieldName": { "type": "keyword" }, "title": { "type": "text" } } }, "visualization": { "properties": { "description": { "type": "text" }, "kibanaSavedObjectMeta": { "properties": { "searchSourceJSON": { "type": "text" } } }, "savedSearchId": { "type": "keyword" }, "title": { "type": "text" }, "uiStateJSON": { "type": "text" }, "version": { "type": "integer" }, "visState": { "type": "text" } } }, "search": { "properties": { "columns": { "type": "keyword" }, "description": { "type": "text" }, "hits": { "type": "integer" }, "kibanaSavedObjectMeta": { "properties": { "searchSourceJSON": { "type": "text" } } }, "sort": { "type": "keyword" }, "title": { "type": "text" }, "version": { "type": "integer" } } }, "dashboard": { "properties": { "description": { "type": "text" }, "hits": { "type": "integer" }, "kibanaSavedObjectMeta": { "properties": { "searchSourceJSON": { "type": "text" } } }, "optionsJSON": { "type": "text" }, "panelsJSON": { "type": "text" }, "refreshInterval": { "properties": { "display": { "type": "keyword" }, "pause": { "type": "boolean" }, "section": { "type": "integer" }, "value": { "type": "integer" } } }, "timeFrom": { "type": "keyword" }, "timeRestore": { "type": "boolean" }, "timeTo": { "type": "keyword" }, "title": { "type": "text" }, "uiStateJSON": { "type": "text" }, "version": { "type": "integer" } } }, "url": { "properties": { "accessCount": { "type": "long" }, "accessDate": { "type": "date" }, "createDate": { "type": "date" }, "url": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 2048 } } } } }, "server": { "properties": { "uuid": { "type": "keyword" } } }, "timelion-sheet": { "properties": { "description": { "type": "text" }, "hits": { "type": "integer" }, "kibanaSavedObjectMeta": { "properties": { "searchSourceJSON": { "type": "text" } } }, "timelion_chart_height": { "type": "integer" }, "timelion_columns": { "type": "integer" }, "timelion_interval": { "type": "keyword" }, "timelion_other_interval": { "type": "keyword" }, "timelion_rows": { "type": "integer" }, "timelion_sheet": { "type": "text" }, "title": { "type": "text" }, "version": { "type": "integer" } } }, "graph-workspace": { "properties": { "description": { "type": "text" }, "kibanaSavedObjectMeta": { "properties": { "searchSourceJSON": { "type": "text" } } }, "numLinks": { "type": "integer" }, "numVertices": { "type": "integer" }, "title": { "type": "text" }, "version": { "type": "integer" }, "wsState": { "type": "text" } } } } } } }'
③將index由.kibana重定向到.kibana-6(reindex.sh)
curl -XPOST '10.0.9.24:9200/_reindex?pretty' -H 'Content-Type: application/json' -d' { "source": { "index": ".kibana" }, "dest": { "index": ".kibana-6" }, "script": { "inline": "ctx._source = [ ctx._type : ctx._source ]; ctx._source.type = ctx._type; ctx._id = ctx._type + \":\" + ctx._id; ctx._type = \"doc\"; ", "lang": "painless" } }'
④將.kibana-6這個index同義爲.kibana,並刪除舊的index,即.kibana(rename.sh)
curl -XPOST '10.0.9.24:9200/_aliases?pretty' -H 'Content-Type: application/json' -d' { "actions" : [ { "add": { "index": ".kibana-6", "alias": ".kibana" } }, { "remove_index": { "index": ".kibana" } } ] }'
4 . 遷移數據目錄data到新版的目錄裏
cp -r /elk/kibana-5.6.1-linux-x86_64/data/* /elk/kibana-6.6.0-linux-x86_64/data
5 . 安裝新版sentinl告警插件
①mv sentinl-v6.6.0.zip /elk/kibana-6.6.0-linux-x86_64/plugins
②cd /elk/kibana-6.6.0-linux-x86_64/plugins
③unzip sentinl-v6.6.0.zip
④mv kibana/sentinl .
⑤rm -rf kibana
6 . 關閉舊進程,啓動新服務
①ps -ef|grep node
②kill -9 pid
③cd /elk/kibana-6.6.0-linux-x86_64/
④./bin/kibana &
三、Logstash升級
1 . 解壓和複製
①tar xvf logstash-6.6.0.tar.gz
②mv logstash-6.6.0 /elk
2 . 配置文件複製
①cd /elk/logstash-6.6.0
②cp /elk/logstash-5.6.1/wls.conf /elk/logstash-6.6.0 #複製配置文件
③cp /elk/logstash-5.6.1/startlogstash /elk/logstash-6.6.0 #複製啓動腳本
④cp /elk/logstash-5.6.1/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-4.1.2/patterns/wls
/elk/logstash-6.6.0/vendor/bundle/jruby/2.3.0/gems/logstash-patterns-core-4.1.2/patterns/ #複製自定義正則文件
3 . 配置文件修改
①vim wls.conf #檢查正則中是否有多餘的空格和回車
②vim startlogstash #修改第一行代碼WorkDir=/elk/logstash-6.6.0
③vim conf/logstash.yml
http.host: "10.0.8.50" xpack.monitoring.enabled: true xpack.monitoring.elasticsearch.url: "http://10.0.9.24:9200" xpack.monitoring.elasticsearch.username: "elastic" xpack.monitoring.elasticsearch.password: "changeme"
④vim conf/jvm..options
-Xms3g -Xmx3g
4 . 關閉舊進程,啓動新服務
①ps -ef|grep logstash
②kill -9 pid
③cd /elk/logstash-6.6.0
④./startlogstash start