ElasticSearch 升級過程中給涉及到的術語-1、同步刷新synced  flush

 Es 會跟蹤每個活動分片的索引,分片在五分鐘內如果沒有收到任何有關索引的操作,則認爲分片不活動了;這將給Es機會去減小分片的資源,同時執行一種特殊刷新,成爲:同步刷新;同步刷新也是執行普通的刷新,然後給所有的分片添加一個不同的標記;

    當沒有索引操作時,id 標記會添加到分片上。標記可以作爲一個快速的方式來檢查兩個分片的lucene索引一致是否一致;這種快速的id 比較主要用於 數據恢復或者重啓後跳過第一個也是成本最高的階段;這種情況下,segment 不需要copy,事務日誌的重演會立即執行;當id 標記和flush 一起用的時候,事物日誌很可能爲空,更加加速了數據的恢復;

這對於具有很多很少更新或者不變的索引的情況(如基於時間的數據),將會更有用;查看分片是否被id 標記,用如下命令:

curl -X GET "dmp9:9201/wzp_test/_stats/commit?level=shards"

{"_shards":{"total":10,"successful":10,"failed":0},"_all":{"primaries":{},"total":{}},"indices":{"wzp_test":{"primaries":{},"total":{},"shards":{"0":[{"routing":{"state":"STARTED","primary":true,"node":"8hkDNjDQRrGMWmhUFbhPrA","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQg==","generation":12,"user_data":{"translog_uuid":"OHqJo56NQoqD0vivoRE_-w","sync_id":"AWdPVMGxhBFTwa1Fywjs","translog_generation":"1543159976461"},"num_docs":411901},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}},{"routing":{"state":"STARTED","primary":false,"node":"aagBamD6QZWOm-GF5QPNdg","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQg==","generation":12,"user_data":{"translog_uuid":"OHqJo56NQoqD0vivoRE_-w","sync_id":"AWdPVMGxhBFTwa1Fywjs","translog_generation":"1543159976461"},"num_docs":411901},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}}],"1":[{"routing":{"state":"STARTED","primary":true,"node":"8hkDNjDQRrGMWmhUFbhPrA","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQQ==","generation":13,"user_data":
{"translog_uuid":"r5RMt3ZDS5ip1Il2cldOKA","sync_id":"AWdPVMGxhBFTwa1Fywjr","translog_generation":"1543160040431"},"num_docs":383147},"shard_path":
{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}},{"routing":{"state":"STARTED","primary":false,"node":"aagBamD6QZWOm-GF5QPNdg","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQQ==","generation":13,"user_data":{"translog_uuid":"r5RMt3ZDS5ip1Il2cldOKA","sync_id":"AWdPVMGxhBFTwa1Fywjr","translog_generation":"1543160040431"},"num_docs":383147},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}}],"2":[{"routing":{"state":"STARTED","primary":true,"node":"8hkDNjDQRrGMWmhUFbhPrA","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAPw==","generation":13,"user_data":{"translog_uuid":"EYkaGyrTTPCtgKArFf0Qyw","sync_id":"AWdPVMGxhBFTwa1Fywjp","translog_generation":"1543159976463"},"num_docs":383110},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}},{"routing":{"state":"STARTED","primary":false,"node":"aagBamD6QZWOm-GF5QPNdg","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAPw==","generation":13,"user_data":{"translog_uuid":"EYkaGyrTTPCtgKArFf0Qyw","sync_id":"AWdPVMGxhBFTwa1Fywjp","translog_generation":"1543159976463"},"num_docs":383110},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}}],"3":[{"routing":{"state":"STARTED","primary":true,"node":"8hkDNjDQRrGMWmhUFbhPrA","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQw==","generation":13,"user_data":{"translog_uuid":"livjpLxlRSakLFnCp8n9Jw","sync_id":"AWdPVMGxhBFTwa1Fywjo","translog_generation":"1543160040432"},"num_docs":410859},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}},{"routing":{"state":"STARTED","primary":false,"node":"aagBamD6QZWOm-GF5QPNdg","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQw==","generation":13,"user_data":{"translog_uuid":"livjpLxlRSakLFnCp8n9Jw","sync_id":"AWdPVMGxhBFTwa1Fywjo","translog_generation":"1543160040432"},"num_docs":410859},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}}],"4":[{"routing":{"state":"STARTED","primary":true,"node":"8hkDNjDQRrGMWmhUFbhPrA","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQA==","generation":13,"user_data":{"translog_uuid":"US1IS1LaR5qkwlJwLLs7Vw","sync_id":"AWdPVMGxhBFTwa1Fywjq","translog_generation":"1543159976461"},"num_docs":416518},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}},{"routing":{"state":"STARTED","primary":false,"node":"aagBamD6QZWOm-GF5QPNdg","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQA==","generation":13,"user_data":{"translog_uuid":"US1IS1LaR5qkwlJwLLs7Vw","sync_id":"AWdPVMGxhBFTwa1Fywjq","translog_generation":"1543159976461"},"num_docs":416518},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}}]}}}}

官方文檔給的結果示例:

curl -X GET "localhost:9200/twitter/_stats/commit?level=shards"

{
   ...

   "indices": {

      "twitter": {

         "primaries": {},

         "total": {},

         "shards": {

            "0": [

               {

                  "routing": {

                     ...
                  },

                  "commit": {

                     "id": "te7zF7C4UsirqvL6jp/vUg==",

                     "generation": 2,

                     "user_data": {

                        "sync_id": "AU2VU0meX-VX2aNbEUsD" ,
                        ...
                     },

                     "num_docs": 0
                  }
               }
               ...
            ],
            ...
         }
     }
   }
}

同步刷新 API

同步刷新API 允許管理員手動刷新。這對於計劃的(回滾)集羣重啓特別有用,您可以停止索引,並且不希望等待默認的5分鐘來自動同步刷新空閒索引;

儘管很方便,但是有一些警告:

  1. 同步刷新是個盡最大努力的操作:任何正在執行索引的操作都會造成同步刷新失敗;這就一位置有些分片可能刷新成功,而有些分片則刷新失敗;

  2. 同步id 的標記在再次刷新是會移除掉;

這是因爲刷新會替換低級別的儲存標記的lucene 提交點;事物日誌中未提交的操作不會移除id 標記;現實中,在任何時間,應該把索引操作視觸發的移除標記爲刷新一樣;

note:有索引操作時,請求同步刷新是無害的;當索引空閒時同步刷新成功,否則失敗;任何請求成功的同步刷新在數據恢復時更加迅速;

curl -X POST "localhost:9200/twitter/_flush/synced"返回值將顯示有多少分片同步刷新成功,及失敗原因;

1)、以下是同步刷新成功的情況;


{
    "_shards": {
    "total": 4,
    "successful": 4,
    "failed": 0
    },
    "twitter": {
    "total": 4,
    "successful": 4,
    "failed": 0
    }
}

2)、以下是失敗情況及其原因;

{

   "_shards": {
      "total": 4,
      "successful": 2,
      "failed": 2
   },
   "twitter": {
      "total": 4,
      "successful": 2,
      "failed": 2,
      "failures": [
         {
            "shard": 1,
            "reason": "[2] ongoing operations on primary"
         }
      ]
   }
}

note:當並行索引操作時,以上錯誤會出現,且http 狀態碼將是 409 ;

3)、有時候分片copy的時候錯誤很特殊;copy失敗對於數據的快速恢復沒有用,但是如果成功了依然有用;具體錯誤如下:

{
   "_shards": {
      "total": 4,
      "successful": 1,
      "failed": 1
   },
   "twitter": {
      "total": 4,
      "successful": 3,
      "failed": 1,
      "failures": [
         {
            "shard": 1,
            "reason": "unexpected error",
            "routing": {
               "state": "STARTED",
               "primary": false,
               "node": "SZNr2J_ORxKTLUCydGX4zA",
               "relocating_node": null,
               "shard": 1,
               "index": "twitter"
            }
         }
      ]
   }
}

 

同步刷新時copy失敗,http 返回碼將是409 ;

4)、同步刷新可以再單一索引上用,也可以在全部索引上用;

單一索引:

POST /kimchy,elasticsearch/_flush/synced

全部索引:

POST /_flush/synced

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章