關於es刪除nodes文件夾後啓動失敗的問題

初次接手kafka+elk,不小心刪除了es的data文件夾下的nodes文件夾。
然後,es健康狀態報紅,我嘗試重啓es集羣,失敗。提取主要報錯,如下:

注:報錯未全部顯示,爲了便於觀察我僅提取主要錯誤展示
報錯1
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.18/n6kDbooRRhuME3I5ntfQcw]]: failed to write index state
報錯2
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.19/MghRcWkvTcydQzJIW1SfbA]]: failed to write index state
報錯3
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.20/MWtk6qR4T9a6IUaJ76dd9A]]: failed to write index state
報錯4
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.21/rwn9gq3PTqe6zNmsBmEDBA]]: failed to write index state
報錯5
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.22/cOwSHci8SwKDhv9Ce33mSg]]: failed to write index state
報錯6
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.23/GwieLpYQTVq-K0I3XO2b_Q]]: failed to write index state
報錯7
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.24/15AcdE3sS0a-5TC-pvw8gg]]: failed to write index state
報錯8
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-kibana-6-2019.03.25/JwtM2CFgQACRDe50yXpdCw]]: failed to write index state
報錯9
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-es-6-2019.03.21/3iEQGvf1T06RZpbrUrmjkw]]: failed to write index state
報錯10
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-es-6-2019.03.22/Vd3rtLeNTX-1huKkQEHtHQ]]: failed to write index state
報錯11
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-es-6-2019.03.23/e_6tPR-qR02L7C0sY4DN1g]]: failed to write index state
報錯12
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-es-6-2019.03.24/RWZshlmBQWe9bEtobpEsWw]]: failed to write index state
報錯13
[WARN ][o.e.g.MetaStateService   ] [es] [[.monitoring-es-6-2019.03.25/CvRjL5UsTcWTSW4t2uhx6g]]: failed to write index state
報錯14
[WARN ][o.e.g.MetaStateService   ] [es] [[.kibana_1/TD4gKls0TuSEvfsSh6zkQw]]: failed to write index state
報錯15
[WARN ][o.e.g.MetaStateService   ] [es] [[.watches/91mld4Y0QoiOYqzxJRzPeA]]: failed to write index state
報錯16
[WARN ][o.e.c.s.ClusterApplierService] [es] failed to notify ClusterStateListener
報錯17
[WARN ][o.e.g.MetaStateService   ] [es] [_global]: failed to write global state

不難發現:錯誤1~15都是在報XXX索引-寫入索引狀態失敗。
所以我就把除了.kibana_1.watches以外的所有索引搜刪掉了
再次啓動,此時只剩下錯誤16,參考https://discuss.elastic.co/t/distress-elasticsearch-does-not-start/152288
我在本地虛擬機搭建了一個全新的es,然後把它的nodes文件夾拷貝過來,然後啓動成功了。
事後我發現並不需要完全拷貝過來,刪掉nodes文件夾後刪掉所有索引,單個啓動,就ok了

**

小結:遇到es集羣報紅,重啓es集羣即可。若刪除了nodes文件,則需要刪除所有索引,然後逐個啓動就ok了。

**

最後,附兩處社區相關問題提問
V2EX:https://www.v2ex.com/t/548200#reply3
stackoverflow: https://stackoverflow.com/questions/55330873/es-failed-to-write-index-state-caused-by-underlying-file-changed-by-an-external

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章