elasticsearch5.2.0和Kibana安装详解及问题

一、es集群的规划准备:
1、修改/etc/hosts文件
192.168.233.131 node1
192.168.233.132 node2
192.168.233.133 node3

临时修改主机名
hostname node1
hostname node2
hostname node3
2、均做免密码登陆配置
ssh-keygen -t rsa
ssh-copy-id -i /root/.ssh/id_rsa.pub node1
ssh-copy-id -i /root/.ssh/id_rsa.pub node2
ssh-copy-id -i /root/.ssh/id_rsa.pub node3
3、编写shell

在node1机器root目录下,产生一个nodes文件,文件内容如下,主要将各节点的主机名维护起来
node02
node03

在node1机器root目录下,产生一个scp.sh文件,文件内容如下,输入需要传送的源文件路径与目标机器的存放路径两个参数,路径都是绝对路径。

#!/bin/bash
cat /root/nodes | while read host
do
{
    scp -r $1 $host:$2
}&wait
done

4.安装jdk1.8,将java_home加入环境变量


5.添加运行elasticsearch的用户(es必须使用非root用户启动)
groupadd elsearch
useradd elsearch -g elsearch -p 123456
chown -R elsearch:elsearch /opt/elasticsearch

6.针对es做的一些系统的优化配置
swapoff -a

echo "fs.file-max = 1000000" >> /etc/sysctl.conf
echo "vm.max_map_count=655360" >> /etc/sysctl.conf
echo "vm.swappiness = 1" >> /etc/sysctl.conf
sysctl -p

sed -i 's/* soft nofile 65535/* soft nofile 655350/g' /etc/security/limits.conf
sed -i 's/* hard nofile 65535/* hard nofile 655350/g' /etc/security/limits.conf


vi /etc/security/limits.d/90-nproc.conf
* soft nproc 2048

7.es内存调整配置文件(建议配置为物理内存的一半或者更多最好不要超过32G,超过了也可能不会增强性能):

/opt/elasticsearch/elasticsearch-5.2.0/config/jvm.options

sed -i 's/-Xms2g/-Xms32g/' /opt/elasticsearch/elasticsearch-5.2.0/config/jvm.options
sed -i 's/-Xmx2g/-Xmx32g/' /opt/elasticsearch/elasticsearch-5.2.0/config/jvm.options
echo "-Xss256k" >>/opt/elasticsearch/elasticsearch-5.2.0/config/jvm.options

sed -i 's/-XX:+UseConcMarkSweepGC/-XX:+UseG1GC/' /opt/elasticsearch/elasticsearch-5.2.0/config/jvm.options

8.集群的主要配置文件
vim  /opt/elasticsearch/elasticsearch-5.2.0/config/elasticsearch.yml


cluster.name: es-cluster
node.name: node1
http.port: 9200
discovery.zen.ping.unicast.hosts: ["172.22.70.211","172.22.70.213","172.22.70.214"]
discovery.zen.minimum_master_nodes: 1
http.cors.enabled: true
http.cors.allow-origin: "*"
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
network.host: 0.0.0.0

9、拷贝文件并启动
sh ~/scp.sh /opt/elasticsearch/ /opt/
sh ~/scp.sh /etc/security/limits.d/90-nproc.conf /etc/security/limits.d/
sh ~/scp.sh /etc/security/limits.conf /etc/security/
sh ~/scp.sh /etc/sysctl.conf /etc/

修改各节点的/opt/elasticsearch/elasticsearch-5.2.0/config/elasticsearch.yml
node.name和端口号

# su - elsearch
cd /opt/elasticsearch/elasticsearch-5.2.0/bin/
或./elasticsearch &
8.检查单台服务是否正常:

$ curl http://ip:9200/
{
  "name" : "yunva_etl_es5",
  "cluster_name" : "yunva-es",
  "cluster_uuid" : "2shAg8u3SjCRNJ4mEUBzBQ",
  "version" : {
    "number" : "5.0.1",
    "build_hash" : "080bb47",
    "build_date" : "2016-11-11T22:08:49.812Z",
    "build_snapshot" : false,
    "lucene_version" : "6.2.1"
  },
  "tagline" : "You Know, for Search"
}


# 查看集群状态
$ curl http://ip:9200/_cluster/health/?pretty
{
  "cluster_name" : "yunva-es",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 5,
  "number_of_data_nodes" : 4,
  "active_primary_shards" : 66,
  "active_shards" : 132,
  "relocating_shards" : 2,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}


然后将配置好的es程序拷贝到其他服务器中,注意修改以下内容(network.host为内网地址,速度更快,节省互相复制、分片的时候处理带宽):
1.elasticsearch.yml文件的配置修改
node.name: 节点名称
network.host: es节点的内网IP地址
2./etc/hosts文件中内网ip和node.name的对应关系

    
三、安装Kibana5.2.0
1、vim kibana/config/kibana.yml
server.port: 5601
server.host: "本机IP"
elasticsearch.url: "http://本机IP:9200"
2、elasticsearch/bin/elasticsearch  //启动ES
kibana/bin/kibana                      //启动kibana
curl -XGET http://192.168.230.150:9200/ 结果如下则成功。

    {
      "name" : "node-1",
      "cluster_name" : "es",
      "cluster_uuid" : "bbCPwel7Tn-1cip2rsFWRQ",
      "version" : {
        "number" : "5.3.1",
        "build_hash" : "5f9cf58",
        "build_date" : "2017-04-17T15:52:53.846Z",
        "build_snapshot" : false,
        "lucene_version" : "6.4.2"
      },
      "tagline" : "You Know, for Search"
    }

浏览器访问9200:http://192.168.230.150:9200可得到上一步同样结果。

浏览器访问5601:http://192.168.230.150:5601.第一次Kibana会进入创建索引的界面,这里创建名为*的索引匹配数据(由于开始没有数据)。(取消掉所有打钩的地方)点击Create成功会显示在左侧。可以看到所有的功能组件。


三、Elasticsearch,Kibana安装中的异常解决:

1、[ WARN]:seccomp unavailable:requires kernel 3.5+ with CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER compiled in。

    警告内核版本太低!,忽略警告。

2、[ERROR]:max number of threads [1024] for user [rzxes] is too low, increase to at least [2048]。

    sudo vim /etc/security/limits.d/90-nproc.conf 把1024改成2048。

3、[ERROR]:max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]。

    Sudo vim /etc/security/limits.conf ,末尾添加如下两行:
    * hard nofile 65536
    * soft nofile 65536
    查看: ulimit -Hn 结果是65536则修改成功。

4、[ERROR]:system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk。

    vim config/elasticsearch.yml 添加如下:
    bootstrap.system_call_filter: false

5、[ERROR]:max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]。

    修改/etc/sysctl.conf sudo vim /etc/sysctl.conf ,最后添加: vm.max_map_count=262144 。
    使用 sysctl -p 查看修改后的结果。
6、[2017-09-15T14:03:19,153][INFO ][o.e.d.z.ZenDiscovery     ] [node3] failed to send join request to master [{node1}{Zo4L-JnrTFq50lsHaH-wsQ}{b4S6zrexQSenOM1bqJCBfg}{172.22.70.211}{172.22.70.211:9300}], reason [RemoteTransportException[[node1][172.22.70.211:9300][internal:discovery/zen/join]]; nested: NotMasterException[Node [{node1}{Zo4L-JnrTFq50lsHaH-wsQ}{b4S6zrexQSenOM1bqJCBfg}{172.22.70.211}{172.22.70.211:9300}] not master for join request]; ], tried [3] times

删除data/ 和 logs/下的所有文件 后重新启动即可,产生如下结果为正确
[2017-09-15T15:35:49,375][INFO ][o.e.c.s.ClusterService   ] [node1] new_master {node1}{VfQ4RZ_eQX-32cTNDCJD1w}{AjZOBuEcQlmy_6yDXorBaQ}{172.22.70.211}{172.22.70.211:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-09-15T15:35:49,436][INFO ][o.e.h.HttpServer         ] [node1] publish_address {172.22.70.211:9200}, bound_addresses {[::]:9200}
[2017-09-15T15:35:49,437][INFO ][o.e.n.Node               ] [node1] started
[2017-09-15T15:35:49,467][INFO ][o.e.g.GatewayService     ] [node1] recovered [0] indices into cluster_state
[2017-09-15T15:38:20,307][INFO ][o.e.c.s.ClusterService   ] [node1] added {{node2}{zs1yhSwZRyqpawOmOI5Wag}{XyOQ-G4ESEizWkh3kGsRPw}{172.22.70.213}{172.22.70.213:9300},}, reason: zen-disco-node-join[{node2}{zs1yhSwZRyqpawOmOI5Wag}{XyOQ-G4ESEizWkh3kGsRPw}{172.22.70.213}{172.22.70.213:9300}]
[2017-09-15T15:38:20,451][WARN ][o.e.d.z.ElectMasterService] [node1] value for setting "discovery.zen.minimum_master_nodes" is too low. This can result in data loss! Please set it to at least a quorum of master-eligible nodes (current value: [1], total number of master-eligible nodes used for publishing in this round: [2])
[2017-09-15T15:39:06,512][INFO ][o.e.c.s.ClusterService   ] [node1] added {{node3}{BvdBNID7QC2nUc3JoWLxSQ}{JL3M1zJyTne_gkkPhNM2Tg}{172.22.70.214}{172.22.70.214:9300},}, reason: zen-disco-node-join[{node3}{BvdBNID7QC2nUc3JoWLxSQ}{JL3M1zJyTne_gkkPhNM2Tg}{172.22.70.214}{172.22.70.214:9300}]




發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章