elasticsearch 核心的http api

開篇

 作爲一個elasticsearch的新手，我覺得elasticsearch的官網的文檔歸類清楚，作爲新手很有必要多閱讀幾次，每次都能加深印象。這裏作爲個人理解進行一些簡單的歸類便於查閱。

 對於新手而言，熟悉各類能夠掌握簡單的操作包括：索引的創建刪除查詢，文檔的增刪改查、集羣信息查詢等。

• Indices APIs：負責索引Index的創建（create）、刪除（delete）、獲取（get）、索引存在（exist）等操作。
• Document APIs：負責索引文檔的創建（index）、刪除（delete）、獲取（get）等操作。
• Search APIs：負責索引文檔的search（查詢），Document APIS根據doc_id進行查詢，Search APIs]根據條件查詢。
• Aggregations：負責針對索引的文檔各維度的聚合（Aggregation）。
• cat APIs：負責查詢索引相關的各類信息查詢。
• Cluster APIs：負責集羣相關的各類信息查詢。

 提供一個操作的方法，就是本機搭建個es集羣然後部署一個kibana，通過kibana熟悉http api就可以了，這篇文章就專注下cat和cluster相關的api。

cat APIs介紹

cat APIs

cat aliases

說明：
aliases 負責展示當前es集羣配置別名包括filter和routing信息。

命令：
GET /_cat/aliases?v
GET /_cat/aliases/alias1,alias2

返回：
alias  index filter routing.index routing.search
alias1 test1 -      -            -
alias2 test1 *      -            -
alias3 test1 -      1            1
alias4 test1 -      2            1,2

cat allocation

說明：
allocation負責展示es的每個數據節點分配的索引分片以及使用的磁盤空間。

命令：
GET /_cat/allocation?v

返回值：
shards disk.indices disk.used disk.avail disk.total disk.percent host      ip        node
     5         260b    47.3gb     43.4gb    100.7gb           46 127.0.0.1 127.0.0.1 CSUXak2

cat count

說明：
count負責展示整個ES集羣或者單個索引的文檔數。

命令：
GET /_cat/count?v
GET /_cat/count/twitter?v

返回值：
epoch      timestamp count
1475868259 15:24:20  120

cat fielddata

說明：
fielddata負責展示ES集羣每個數據節點中fileddata佔用的堆內存。

命令：
GET /_cat/fielddata?v
GET /_cat/fielddata?v&fields=body
GET /_cat/fielddata/body,soul?v

返回值：
id                     host      ip        node    field   size
Nqk-6inXQq-OxUfOUI8jNQ 127.0.0.1 127.0.0.1 Nqk-6in body    544b
Nqk-6inXQq-OxUfOUI8jNQ 127.0.0.1 127.0.0.1 Nqk-6in soul    480b

cat health

說明：
health負責展示集羣的健康狀態

命令：
GET /_cat/health?v
GET /_cat/health?v&ts=false

返回值：
epoch      timestamp cluster       status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1475871424 16:17:04  elasticsearch green           1         1      5   5    0    0        0             0                  -                100.0%

cat indices

說明：
indices負責提供索引的相關信息，包括組成一個索引（index）的shard、document的數量，刪除的doc數量，主存大小和所有索引的總存儲大小。

命令：
GET /_cat/indices/twi*?v&s=index


返回值：
health status index    uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   twitter  u8FNjxh8Rfy_awN11oDKYQ   1   1       1200            0     88.1kb         88.1kb
green  open   twitter2 nYFWZEO7TUiOjLQXBaYJpA   5   0          0            0       260b           260b

cat master

說明：
master負責展示es集羣的master節點信息包括節點id、節點名、ip地址等。

命令：
GET /_cat/master?v

返回值：
id                     host      ip        node
YzWoH_2BT-6UjVGDyPdqYg 127.0.0.1 127.0.0.1 YzWoH_2

cat nodeattrs

說明：
nodeattrs負責展示通用的節點信息。

命令：
GET /_cat/nodeattrs?v

返回值：
node    host      ip        attr     value
node-0 127.0.0.1 127.0.0.1 testattr test

cat nodes

說明：
nodes負責展示es集羣的拓撲信息。

命令：
GET /_cat/nodes?v

返回值：
ip        heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
127.0.0.1           65          99  42    3.07                  mdi       *      mJw06l1

cat pending tasks

說明：
pending_tasks返回集羣層面的未執行的任務列表包括創建索引，更新mapping，allocate分片信息。

命令：
GET /_cat/pending_tasks?v

返回值：
insertOrder timeInQueue priority source
       1685       855ms HIGH     update-mapping [foo][t]
       1686       843ms HIGH     update-mapping [foo][t]
       1693       753ms HIGH     refresh-mapping [foo][[t]]
       1688       816ms HIGH     update-mapping [foo][t]
       1689       802ms HIGH     update-mapping [foo][t]
       1690       787ms HIGH     update-mapping [foo][t]
       1691       773ms HIGH     update-mapping [foo][t]

cat plugins

說明：
plugins命令展示每個節點正在運行的插件信息。

命令：
GET /_cat/plugins?v&s=component&h=name,component,version,description

返回值：
name    component               version   description
U7321H6 analysis-icu            6.5.4 The ICU Analysis plugin integrates Lucene ICU module into elasticsearch, adding ICU relates analysis components.
U7321H6 analysis-kuromoji       6.5.4 The Japanese (kuromoji) Analysis plugin integrates Lucene kuromoji analysis module into elasticsearch.
U7321H6 analysis-nori           6.5.4 The Korean (nori) Analysis plugin integrates Lucene nori analysis module into elasticsearch.
U7321H6 analysis-phonetic       6.5.4 The Phonetic Analysis plugin integrates phonetic token filter analysis with elasticsearch.

cat recovery

說明：
recovery命令展示索引分片恢復，包括正在進行的和已經完成的任務。

命令：
GET _cat/recovery?v
GET _cat/recovery?v&h=i,s,t,ty,st,shost,thost,f,fp,b,bp

返回值：
index   shard time type  stage source_host source_node target_host target_node repository snapshot files files_recovered files_percent files_total bytes bytes_recovered bytes_percent bytes_total translog_ops translog_ops_recovered translog_ops_percent
twitter 0     13ms store done  n/a         n/a         127.0.0.1   node-0      n/a        n/a      0     0               100%          13          0     0               100%          9928        0            0                      100.0%

cat repositories

說明：
repositories命令展示在集羣中註冊的快照倉庫。

命令：
GET /_cat/repositories?v
GET /_cat/thread_pool/generic?v&h=id,name,active,rejected,completed

返回值：
id    type
repo1   fs
repo2   s3

cat thread pool

說明：
thread_pool命令展示集羣節點的線程池的統計信息，包括處於活躍、隊列等待、拒絕的線程任務信息。

命令：
GET /_cat/thread_pool

返回值：
node-0 bulk                0 0 0
node-0 fetch_shard_started 0 0 0
node-0 fetch_shard_store   0 0 0
node-0 flush               0 0 0
node-0 force_merge         0 0 0
node-0 generic             0 0 0
node-0 get                 0 0 0
node-0 index               0 0 0
node-0 listener            0 0 0
node-0 management          1 0 0
node-0 refresh             0 0 0
node-0 search              0 0 0
node-0 snapshot            0 0 0
node-0 warmer              0 0 0

cat shards

說明：
shards命令展示每個節點包括哪些分片信息。

命令：
GET _cat/shards
GET _cat/shards/twitt*

返回值：
twitter 0 p STARTED 3014 31.1mb 192.168.56.10 H5dfFeA

cat segments

說明：
segments命令展示索引的segments的信息。 

命令：
GET /_cat/segments?v

返回值：
index shard prirep ip        segment generation docs.count docs.deleted size size.memory committed searchable version compound
test  3     p      127.0.0.1 _0               0          1            0  3kb        2042 false     true       7.1.0   true
test1 3     p      127.0.0.1 _0               0          1            0  3kb        2042 false     true       7.1.0   true

cat snapshots

說明：
snapshots命令展示屬於某個指定快照倉庫的所有快照信息。通過/_cat/repositories查找所有的倉庫列表。

命令：
GET /_cat/snapshots/repo1?v&s=id

返回值：
id     status start_epoch start_time end_epoch  end_time duration indices successful_shards failed_shards total_shards
snap1  FAILED 1445616705  18:11:45   1445616978 18:16:18     4.6m       1                 4             1            5
snap2 SUCCESS 1445634298  23:04:58   1445634672 23:11:12     6.2m       2                10             0           10

cat templates

說明：
templates展示已存在的模板信息。

命令：
GET /_cat/templates?v&s=name

返回值：
name      index_patterns order version
template0 [te*]          0
template1 [tea*]         1
template2 [teak*]        2     7

Cluster APIs介紹

Cluster APIs
Cluster Health

說明：
health命令展示集羣的健康狀態，

命令：
GET _cluster/health
GET /_cluster/health/test1,test2

返回值：
{
  "cluster_name" : "testcluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 5,
  "active_shards" : 5,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 5,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 50.0
}

Cluster State

說明：
state命令展示集羣詳細的狀態信息，
其中metrics指標包括version、master_node、nodes、routing_table、metadata、blocks。

命令：
GET /_cluster/state
GET /_cluster/state/{metrics}/{indices}
GET /_cluster/state/metadata,routing_table/foo,bar

返回值：
{
  "cluster_name": "my-application",
  "compressed_size_in_bytes": 847,
  "version": 10,
  "state_uuid": "DDWoFtm7TvCydbp8SKRQdw",
  "master_node": "YOIwJR1bTxG0iKHur_M-Rg"
}

Cluster Stats

說明：
Stats命令展示集羣的多維度統計信息。
指標包括shard numbers, store size, memory usage、
number, roles, os, jvm versions, memory usage, cpu and installed plugins。

命令：
GET /_cluster/stats?human&pretty

返回值：
{
   "_nodes" : {
      "total" : 1,
      "successful" : 1,
      "failed" : 0
   },
   "cluster_name": "elasticsearch",
   "timestamp": 1459427693515,
   "status": "green",
   "indices": {
      "count": 1,
      "shards": {
         "total": 5,
         "primaries": 5,
         "replication": 0,
         "index": {
            "shards": {
               "min": 5,
               "max": 5,
               "avg": 5
            },
            "primaries": {
               "min": 5,
               "max": 5,
               "avg": 5
            },
            "replication": {
               "min": 0,
               "max": 0,
               "avg": 0
            }
         }
      },
      "docs": {
         "count": 10,
         "deleted": 0
      },
      "store": {
         "size": "16.2kb",
         "size_in_bytes": 16684
      },
      "fielddata": {
         "memory_size": "0b",
         "memory_size_in_bytes": 0,
         "evictions": 0
      },
      "query_cache": {
         "memory_size": "0b",
         "memory_size_in_bytes": 0,
         "total_count": 0,
         "hit_count": 0,
         "miss_count": 0,
         "cache_size": 0,
         "cache_count": 0,
         "evictions": 0
      },
      "completion": {
         "size": "0b",
         "size_in_bytes": 0
      },
      "segments": {
         "count": 4,
         "memory": "8.6kb",
         "memory_in_bytes": 8898,
         "terms_memory": "6.3kb",
         "terms_memory_in_bytes": 6522,
         "stored_fields_memory": "1.2kb",
         "stored_fields_memory_in_bytes": 1248,
         "term_vectors_memory": "0b",
         "term_vectors_memory_in_bytes": 0,
         "norms_memory": "384b",
         "norms_memory_in_bytes": 384,
         "points_memory" : "0b",
         "points_memory_in_bytes" : 0,
         "doc_values_memory": "744b",
         "doc_values_memory_in_bytes": 744,
         "index_writer_memory": "0b",
         "index_writer_memory_in_bytes": 0,
         "version_map_memory": "0b",
         "version_map_memory_in_bytes": 0,
         "fixed_bit_set": "0b",
         "fixed_bit_set_memory_in_bytes": 0,
         "max_unsafe_auto_id_timestamp" : -9223372036854775808,
         "file_sizes": {}
      }
   },
   "nodes": {
      "count": {
         "total": 1,
         "data": 1,
         "coordinating_only": 0,
         "master": 1,
         "ingest": 1
      },
      "versions": [
         "6.1.4"
      ],
      "os": {
         "available_processors": 8,
         "allocated_processors": 8,
         "names": [
            {
               "name": "Mac OS X",
               "count": 1
            }
         ],
         "mem" : {
            "total" : "16gb",
            "total_in_bytes" : 17179869184,
            "free" : "78.1mb",
            "free_in_bytes" : 81960960,
            "used" : "15.9gb",
            "used_in_bytes" : 17097908224,
            "free_percent" : 0,
            "used_percent" : 100
         }
      },
      "process": {
         "cpu": {
            "percent": 9
         },
         "open_file_descriptors": {
            "min": 268,
            "max": 268,
            "avg": 268
         }
      },
      "jvm": {
         "max_uptime": "13.7s",
         "max_uptime_in_millis": 13737,
         "versions": [
            {
               "version": "1.8.0_74",
               "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
               "vm_version": "25.74-b02",
               "vm_vendor": "Oracle Corporation",
               "count": 1
            }
         ],
         "mem": {
            "heap_used": "57.5mb",
            "heap_used_in_bytes": 60312664,
            "heap_max": "989.8mb",
            "heap_max_in_bytes": 1037959168
         },
         "threads": 90
      },
      "fs": {
         "total": "200.6gb",
         "total_in_bytes": 215429193728,
         "free": "32.6gb",
         "free_in_bytes": 35064553472,
         "available": "32.4gb",
         "available_in_bytes": 34802409472
      },
      "plugins": [
        {
          "name": "analysis-icu",
          "version": "6.1.4",
          "description": "The ICU Analysis plugin integrates Lucene ICU module into elasticsearch, adding ICU relates analysis components.",
          "classname": "org.elasticsearch.plugin.analysis.icu.AnalysisICUPlugin",
          "has_native_controller": false
        },
        {
          "name": "ingest-geoip",
          "version": "6.1.4",
          "description": "Ingest processor that uses looksup geo data based on ip adresses using the Maxmind geo database",
          "classname": "org.elasticsearch.ingest.geoip.IngestGeoIpPlugin",
          "has_native_controller": false
        },
        {
          "name": "ingest-user-agent",
          "version": "6.1.4",
          "description": "Ingest processor that extracts information from a user agent",
          "classname": "org.elasticsearch.ingest.useragent.IngestUserAgentPlugin",
          "has_native_controller": false
        }
      ],
      "network_types" : {
         "transport_types" : {
            "netty4" : 1
         },
         "http_types" : {
            "netty4" : 1
         }
      }
   }
}

Pending cluster tasks

說明：
pending_tasks返回集羣級別的未執行的任務，
包括創建索引,、更新mapping、 分配分片等。

命令：
ET /_cluster/pending_tasks

返回值：
{
   "tasks": [
      {
         "insert_order": 101,
         "priority": "URGENT",
         "source": "create-index [foo_9], cause [api]",
         "time_in_queue_millis": 86,
         "time_in_queue": "86ms"
      },
      {
         "insert_order": 46,
         "priority": "HIGH",
         "source": "shard-started ([foo_2][1], node[tMTocMvQQgGCkj7QDHl3OA], [P], s[INITIALIZING]), reason [after recovery from shard_store]",
         "time_in_queue_millis": 842,
         "time_in_queue": "842ms"
      },
      {
         "insert_order": 45,
         "priority": "HIGH",
         "source": "shard-started ([foo_2][0], node[tMTocMvQQgGCkj7QDHl3OA], [P], s[INITIALIZING]), reason [after recovery from shard_store]",
         "time_in_queue_millis": 858,
         "time_in_queue": "858ms"
      }
  ]
}

Cluster Reroute

說明：
reroute命令允許我們人工的重新分配集羣中的索引分片。
move：移動分片
cancel：取消分片分配
allocate_replica：分配副本

命令：
POST /_cluster/reroute
{
    "commands" : [
        {
            "move" : {
                "index" : "test", "shard" : 0,
                "from_node" : "node1", "to_node" : "node2"
            }
        },
        {
          "allocate_replica" : {
                "index" : "test", "shard" : 1,
                "node" : "node3"
          }
        }
    ]
}

Cluster Update Settings

說明：
更新集羣的配置信息並返回執行更新完成的配置。

命令：
PUT /_cluster/settings
{
    "persistent" : {
        "indices.recovery.max_bytes_per_sec" : "50mb"
    }
}

PUT /_cluster/settings?flat_settings=true
{
    "transient" : {
        "indices.recovery.max_bytes_per_sec" : "20mb"
    }
}

返回值：
{
    ...
    "persistent" : { },
    "transient" : {
        "indices.recovery.max_bytes_per_sec" : "20mb"
    }
}

Nodes Stats

說明：
_nodes/stats 展示集羣中指定節點或所有節點的統計信息。

命令：
GET /_nodes/stats
GET /_nodes/nodeId1,nodeId2/stats

返回值：
{
  "_nodes": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "cluster_name": "my-application",
  "nodes": {
    "YOIwJR1bTxG0iKHur_M-Rg": {
      "timestamp": 1545564028595,
      "name": "node-2",
      "transport_address": "127.0.0.1:9300",
      "host": "127.0.0.1",
      "ip": "127.0.0.1:9300",
      "roles": [
        "master",
        "data",
        "ingest"
      ],
      "os": {
        "timestamp": 1545564028595,
        "cpu": {
          "percent": 12,
          "load_average": {
            "1m": 1.7255859375
          }
        },
        "mem": {
          "total_in_bytes": 8589934592,
          "free_in_bytes": 305328128,
          "used_in_bytes": 8284606464,
          "free_percent": 4,
          "used_percent": 96
        },
        "swap": {
          "total_in_bytes": 2147483648,
          "free_in_bytes": 1657012224,
          "used_in_bytes": 490471424
        }
      }
    }
  }
}

Nodes Info

說明：
_nodes展示集羣中每個節點的信息。

命令：
GET /_nodes
GET /_nodes/nodeId1,nodeId2
指標包括settings, os, process, jvm, thread_pool, transport, http, plugins, ingest，indices

返回值：
{
  "_nodes": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "cluster_name": "my-application",
  "nodes": {
    "YOIwJR1bTxG0iKHur_M-Rg": {
      "name": "node-2",
      "transport_address": "127.0.0.1:9300",
      "host": "127.0.0.1",
      "ip": "127.0.0.1",
      "version": "6.1.0",
      "build_hash": "c0c1ba0",
      "roles": [
        "master",
        "data",
        "ingest"
      ],
      "os": {
        "refresh_interval_in_millis": 1000,
        "name": "Mac OS X",
        "arch": "x86_64",
        "version": "10.14.2",
        "available_processors": 4,
        "allocated_processors": 4
      }
    }
  }
}

Nodes Feature Usage

說明：
_nodes/usage展示集羣各個節點各類服務調用次數

命令：
GET _nodes/usage
GET _nodes/nodeId1,nodeId2/usage

返回值：
{
  "_nodes": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "cluster_name": "my_cluster",
  "nodes": {
    "pQHNt5rXTTWNvUgOrdynKg": {
      "timestamp": 1492553961812, 
      "since": 1492553906606, 
      "rest_actions": {
        "org.elasticsearch.rest.action.admin.cluster.RestNodesUsageAction": 1,
        "org.elasticsearch.rest.action.admin.indices.RestCreateIndexAction": 1,
        "org.elasticsearch.rest.action.document.RestGetAction": 1,
        "org.elasticsearch.rest.action.search.RestSearchAction": 19, 
        "org.elasticsearch.rest.action.admin.cluster.RestNodesInfoAction": 36
      }
    }
  }
}

Remote Cluster Info

說明：
返回集羣的連接信息。

命令：
GET /_remote/info

Nodes hot_threads

說明：
_nodes/hot_threads展示集羣節點的熱點線程信息。

命令：
GET /_nodes/hot_threads
GET /_nodes/{nodesIds}/hot_threads.

返回值：
::: {node-2}{YOIwJR1bTxG0iKHur_M-Rg}{UmVpL8zQTUa1tu6TtOhpyQ}{127.0.0.1}{127.0.0.1:9300}
   Hot threads at 2018-12-23T12:22:24.368Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:

Cluster Allocation Explain API

說明：
用於解釋分片沒有被分配的原因

命令：
GET /_cluster/allocation/explain
{
  "index": "myindex",
  "shard": 0,
  "primary": true
}

返回值：
{
  "index" : "idx",
  "shard" : 0,
  "primary" : true,
  "current_state" : "unassigned",                 
  "unassigned_info" : {
    "reason" : "INDEX_CREATED",                   
    "at" : "2017-01-04T18:08:16.600Z",
    "last_allocation_status" : "no"
  },
  "can_allocate" : "no",                          
  "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",
  "node_allocation_decisions" : [
    {
      "node_id" : "8qt2rY-pT6KNZB3-hGfLnw",
      "node_name" : "node-0",
      "transport_address" : "127.0.0.1:9401",
      "node_attributes" : {},
      "node_decision" : "no",                     
      "weight_ranking" : 1,
      "deciders" : [
        {
          "decider" : "filter",                   
          "decision" : "NO",
          "explanation" : "node does not match index setting [index.routing.allocation.include] filters [_name:\"non_existent_node\"]"  
        }
      ]
    }
  ]
}