Elastic Certified Engineer複習記錄-集羣搭建docker篇

原創 死敌wen 2020-07-06 13:10

文章目錄


之前提到集羣搭建的時候可以通過多種方式，考試的時候多半使用的是rpm或者tar包的方式，但是在沒有足夠多資源的時候（囊中羞澀又想白嫖），用docker（docker-compose/k8s）的方式搭建對我們來說是最友好的。

環境準備

安裝包括docker、docker-compose以及相關的軟件。

brew update
brew install docker
brew install docker-compose

下載相關的docker鏡像

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.6.0
docker pull docker.elastic.co/kibana/kibana:7.6.0

這裏沒有用oss版（純開源版）主要是爲了後面可能會有xpack相關的配置的練習，如果用oss版的話可能xpack的組件整個都沒了。

這裏注意kibana的版本要和es一致，至少大版本是一致的（7.6.x配7.6.x）否則輕則啓動之後不停的報錯，重則直接因爲版本不兼容而啓動失敗。

創建一個專用的網絡

以避免和本機其他docker組件的網絡有衝突。
docker network create bigdata

開始編寫docker-compose 文件

既然考試的初衷是模擬真實的使用場景，我們可以考慮先搭建一個多節點（此處是3節點）的es集羣。（完整的docker-compose文件後面會添上）

配置組合集羣，主要有幾個關鍵配置
1. cluster.name=docker-cluster 聲明集羣名稱
2. discovery.seed_hosts=node2,node3 集羣初始化的時候需要彼此保活的節點
3. cluster.initial_master_nodes=node1初始化時候的master節點
4. "ES_JAVA_OPTS=-Xms2g -Xmx2g" es啓動的最大/小內存，官方默認配置是512m
5. 內存限制
6. （可選）bootstrap.memory_lock=true 交換區鎖定
7. （可選）esdata01:/usr/share/elasticsearch/data文件存儲掛載

其他的包括nodename、開放端口之類的都是docker的常規操作，不在這裏詳細介紹。

最簡版docker-compose.yml

version: '3.6'
networks:
  bigdata:
    external: true // 專用網絡

volumes:
  esdata01: // 磁盤掛載，主要是data文件
    driver: local
  esdata02:
    driver: local
  esdata03:
    driver: local
    
services:
  node1: // 單一節點配置
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0 // 鏡像
    container_name: node1 // 節點名稱，其實可以不寫
    environment: // 上面提到的那些環境參數
      - node.name=node1 
      - cluster.name=docker-cluster
      - discovery.seed_hosts=node2,node3
      - cluster.initial_master_nodes=node1
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01:/usr/share/elasticsearch/data
    ports: // 開放端口
      - 9200:9200
      - 9300:9300
    networks: // 使用專用網絡組
      - bigdata

  node2:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
    container_name: node2
    environment:
      - node.name=node2
      - cluster.name=docker-cluster
      - discovery.seed_hosts=node1,node3
      - cluster.initial_master_nodes=node1
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata02:/usr/share/elasticsearch/data
    networks:
      - bigdata

  node3:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
    container_name: node3
    environment:
      - node.name=node3
      - cluster.name=docker-cluster
      - discovery.seed_hosts=node1,node2
      - cluster.initial_master_nodes=node1
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata03:/usr/share/elasticsearch/data
    networks:
      - bigdata

  kibana:
    image: docker.elastic.co/kibana/kibana:7.6.0
    container_name: kibana
    environment:
      ELASTICSEARCH_HOSTS: http://node1:9200
    depends_on:
      - node1
      - node2
      - node3
    external_links:
      - node1
      - node2
      - node3
    networks:
      - bigdata
    ports:
      - 5601:5601

可以通過命令docker-compose -f "${filepath}/docker-compose.yml" up -d --build來編譯啓動

➜ docker docker-compose -f "docker-elasticsearch/docker-compose.yml" up -d --build
Creating node1 ... done
Creating node3 ... done
Creating node2 ... done
Creating kibana ... done

然後通過命令docker ps -as 查看啓動狀態

➜ docker ps -as
CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                                            NAMES               SIZE
cc9527af7fee        docker.elastic.co/kibana/kibana:7.6.0                 "/usr/local/bin/dumb…"   5 hours ago         Up 5 hours          0.0.0.0:5601->5601/tcp                           kibana              135MB (virtual 1.14GB)
0641c9015768        docker.elastic.co/elasticsearch/elasticsearch:7.6.0   "/usr/local/bin/dock…"   16 hours ago        Up 16 hours         9200/tcp, 9300/tcp                               node3               3.47MB (virtual 794MB)
0b1f7b4ae9c1        docker.elastic.co/elasticsearch/elasticsearch:7.6.0   "/usr/local/bin/dock…"   16 hours ago        Up 16 hours         0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   node1               5.92MB (virtual 796MB)
a165b7826a35        docker.elastic.co/elasticsearch/elasticsearch:7.6.0   "/usr/local/bin/dock…"   16 hours ago        Up 16 hours         9200/tcp, 9300/tcp                               node2               4.59MB (virtual 795MB)

嘗試打開xpack相關安全性配置

這裏有個可能會讓人很困惑的地方，如果我們在docker-compose文件裏只開啓xpack的安全性認證xpack.security.enabled: true，啓動的時候集羣會報錯，提示需要把xpack.security.transport.ssl.enabled一併開啓，否則就需要關掉xpack的認證。如果我們把ssl的配置開啓了之後，集羣啓動的時候又會報認證失敗的錯。

那就讓我們一步步解決這些東西。

  1. 關閉那倆配置，讓集羣正常啓動
……省略一些
services:
  node1: 
    environment:
	  - xpack.security.enabled: false
	  - xpack.security.transport.ssl.enabled: false
……其他的省略
  1. 登入其中一個節點，創建認證中心（certificate authority）
    進入docker節點 docker exec -it node1 bash
[root@1b2b7e0c0734 elasticsearch]# pwd
/usr/share/elasticsearch

用es工具生成證書 ./bin/elasticsearch-certutil ca

[root@1b2b7e0c0734 elasticsearch]# ./bin/elasticsearch-certutil ca
WARNING: An illegal reflective access operation has occurred
……中間一大堆……
Please enter the desired output file [elastic-stack-ca.p12]: // 這一行是指定認證機構文件生成路徑，不填默認當前路徑
Enter password for elastic-stack-ca.p12 : // 這一行是指定ca密碼，不填爲空

檢查一下結果

[root@1b2b7e0c0734 elasticsearch]# ls -ltr
total 572
-rw-r--r--  1 elasticsearch root   8164 Feb  6 00:07 README.asciidoc
……一些ES自己的文件……
drwxrwxr-x  1 elasticsearch root   4096 Mar 24 16:34 config
-rw-------  1 root          root   2527 Mar 28 15:45 elastic-stack-ca.p12  <----要的就是它
  1. 創建認證文件 bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12（這文件不在當前路徑的話要指定它的路徑）
[root@1b2b7e0c0734 elasticsearch]# ./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
WARNING: An illegal reflective access operation has occurred
……中間一大堆…… // 指定證書密碼生成的文件密碼等等，最終會被生成到下面那個路徑
Enter password for CA (elastic-stack-ca.p12) : 
Please enter the desired output file [elastic-certificates.p12]:
Enter password for elastic-certificates.p12 :

Certificates written to /usr/share/elasticsearch/elastic-certificates.p12
……中間一大堆……
For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.

檢查一下

[root@1b2b7e0c0734 elasticsearch]# ls -ltr
total 576
-rw-r--r--  1 elasticsearch root   8164 Feb  6 00:07 README.asciidoc
……一些ES自己的文件……
drwxrwxr-x  1 elasticsearch root   4096 Mar 24 16:34 config
-rw-------  1 root          root   2527 Mar 28 15:45 elastic-stack-ca.p12 
-rw-------  1 root          root   3443 Mar 28 15:52 elastic-certificates.p12 <--- 多了一個它
  1. docker節點之間不能直接拷文件有點煩人（不是完全不可以，不過過程很曲折），只能先把這個認證文件拷到宿主機上，然後再從宿主機拷到其他節點裏。不過我們可以直接通過宿主機掛載的方式，以宿主機爲介質讓所有節點共享這個文件。
    1. 先把文件拷到宿主機docker cp node1:/usr/share/elasticsearch/elastic-certificates.p12 .
    2. 在所有節點的掛載配置上加一行- ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
services:
  node1:
    volumes:
    - ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12 // 加這行
  1. 這時就可以開啓1.裏面的兩個配置，讓集羣正常啓動。但是此時，我們通過kibana登陸的時候會要求輸入賬號密碼。
  2. 我們來給集羣創建一組密碼，方便我們通過不同身份進行登陸和集羣的操作。
(宿主機上登陸某個幾點) docker exec -it node1 bash

[root@9a7aeeb9be4d elasticsearch]# ./bin/elasticsearch-setup-passwords auto // 自動創建所有用戶及密鑰，也可以通過手動方式指定
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y


Changed password for user apm_system
PASSWORD apm_system = 9SMEwVztnQT3EkTPaQ7X

Changed password for user kibana
PASSWORD kibana = I28UJQgCoMUDM2SPjyu9

Changed password for user logstash_system
PASSWORD logstash_system = KNlDRpZpdSqFyaKjyiy2

Changed password for user beats_system
PASSWORD beats_system = U6vajAbRBI5RwX00CYuv

Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = ez2eNRJty1ACp18cv5Wy

Changed password for user elastic
PASSWORD elastic = 5RgiAQSCvGyHZdW5EsYy

至此，我們就能夠正常的啓動3es+1kibana節點的docker集羣了。
查看一下

// 啓動集羣
➜ ✗ docker-compose -f "docker-elasticsearch/docker-compose.yml" up -d --build
// 查看docker節點（們）的狀態
➜ ✗ docker ps -as
CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                                            NAMES               SIZE
cc9527af7fee        docker.elastic.co/kibana/kibana:7.6.0                 "/usr/local/bin/dumb…"   5 hours ago         Up 5 hours          0.0.0.0:5601->5601/tcp                           kibana              135MB (virtual 1.14GB)
0641c9015768        docker.elastic.co/elasticsearch/elasticsearch:7.6.0   "/usr/local/bin/dock…"   16 hours ago        Up 16 hours         9200/tcp, 9300/tcp                               node3               3.7MB (virtual 794MB)
0b1f7b4ae9c1        docker.elastic.co/elasticsearch/elasticsearch:7.6.0   "/usr/local/bin/dock…"   16 hours ago        Up 16 hours         0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   node1               6.15MB (virtual 796MB)
a165b7826a35        docker.elastic.co/elasticsearch/elasticsearch:7.6.0   "/usr/local/bin/dock…"   16 hours ago        Up 16 hours         9200/tcp, 9300/tcp                               node2               4.66MB (virtual 795MB)
// 查看es集羣狀態
➜ ✗ curl http://elastic:5RgiAQSCvGyHZdW5EsYy@localhost:9200/ | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   534  100   534    0     0  38142      0 --:--:-- --:--:-- --:--:-- 38142
{
  "name": "node1",
  "cluster_name": "docker-cluster",
  "cluster_uuid": "wZFXKEITRKWVg36vUHWgyQ",
  "version": {
    "number": "7.6.0",
    "build_flavor": "default",
    "build_type": "docker",
    "build_hash": "7f634e9f44834fbc12724506cc1da681b0c3b1e3",
    "build_date": "2020-02-06T00:09:00.449973Z",
    "build_snapshot": false,
    "lucene_version": "8.4.0",
    "minimum_wire_compatibility_version": "6.8.0",
    "minimum_index_compatibility_version": "6.0.0-beta1"
  },
  "tagline": "You Know, for Search"
}

頁面查看集羣狀態
#由於我們開啓了密碼驗證，所以簡單的通過localhost:9200來查看集羣狀態的話會報安全認證錯誤

➜ elasticsearch git:(7.6) ✗ curl http://localhost:9200/ | jq .
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   381  100   381    0     0  22411      0 --:--:-- --:--:-- --:--:-- 22411
{
  "error": {
    "root_cause": [
      {
        "type": "security_exception",
        "reason": "missing authentication credentials for REST request [/]",
        "header": {
          "WWW-Authenticate": "Basic realm=\"security\" charset=\"UTF-8\""
        }
      }
    ],
    "type": "security_exception",
    "reason": "missing authentication credentials for REST request [/]",
    "header": {
      "WWW-Authenticate": "Basic realm=\"security\" charset=\"UTF-8\""
    }
  },
  "status": 401
}

完整的docker-compose.yml文件

version: '3.6'
networks:
  bigdata:
    external: true

volumes:
  esdata01:
    driver: local
  esdata02:
    driver: local
  esdata03:
    driver: local
    
services:
  node1:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
    container_name: node1
    environment:
      - node.name=node1
      - cluster.name=docker-cluster
      - discovery.seed_hosts=node2,node3
      - cluster.initial_master_nodes=node1
      - xpack.security.enabled=true
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.keystore.type=PKCS12
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.type=PKCS12
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata01:/usr/share/elasticsearch/data
      - ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
    ports:
      - 9200:9200
      - 9300:9300
    networks:
      - bigdata

  node2:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
    container_name: node2
    environment:
      - node.name=node2
      - cluster.name=docker-cluster
      - discovery.seed_hosts=node1,node3
      - cluster.initial_master_nodes=node1
      - xpack.security.enabled=true
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.keystore.type=PKCS12
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.type=PKCS12
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata02:/usr/share/elasticsearch/data
      - ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
    networks:
      - bigdata

  node3:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
    container_name: node3
    environment:
      - node.name=node3
      - cluster.name=docker-cluster
      - discovery.seed_hosts=node1,node2
      - cluster.initial_master_nodes=node1
      - xpack.security.enabled=true
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.keystore.type=PKCS12
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.keystore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.path=elastic-certificates.p12
      - xpack.security.transport.ssl.truststore.type=PKCS12
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms2g -Xmx2g"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata03:/usr/share/elasticsearch/data
      - ./elastic-certificates.p12:/usr/share/elasticsearch/config/elastic-certificates.p12
    networks:
      - bigdata

  kibana:
    image: docker.elastic.co/kibana/kibana:7.6.0
    container_name: kibana
    environment:
      ELASTICSEARCH_HOSTS: http://node3:9200
      XPACK_MONITORING_ENABLED: "true"
      ELASTICSEARCH_USERNAME: kibana
      ELASTICSEARCH_PASSWORD: I28UJQgCoMUDM2SPjyu9
      SERVER_HOST: 0.0.0.0
    depends_on:
      - node1
      - node2
      - node3
    external_links:
      - node1
      - node2
      - node3
    networks:
      - bigdata
    ports:
      - 5601:5601

這裏還有個坑點，我們開啓了密碼驗證之後，在kibana的配置當中需要添加es的username和password。

      ELASTICSEARCH_USERNAME: kibana
      ELASTICSEARCH_PASSWORD: I28UJQgCoMUDM2SPjyu9

分別對應kibana.yml裏面的

elasticsearch.username: kibana
elasticsearch.password: changeme

但是當我們通過web頁面登陸的時候，輸入的賬號密碼不是kibana，而是elastic這個賬號的。
賬號密碼登陸
用kibana登陸的時候
kibana登陸
拒絕登陸
這是因爲kibana這個賬號是給kibana節點自己做認證用的，不是給用戶用的，我們需要用elastic這個賬號登陸，這個纔是人類管理員的賬號。

// 不能用這倆
Changed password for user kibana
PASSWORD kibana = I28UJQgCoMUDM2SPjyu9
// 要用這倆
Changed password for user elastic
PASSWORD elastic = 5RgiAQSCvGyHZdW5EsYy

集羣狀態
pls enjoy

重點總結

  1. 安裝docker相關軟件
  2. 關閉xpack認證啓動集羣
  3. 登陸任意一個es節點，創建認證文件
  4. 創建不同賬戶及密碼
  5. 開啓所有安全認證，掛載ca文件，啓動集羣
  6. 登陸成功並繼續後續操作

參考文獻：

Running the Elastic Stack on Docker
Setting up Elasticsearch and Kibana on Docker with X-Pack security enabled
Configuring Kibana
Install Elasticsearch with Docker
Security settings in Elasticsearch
Encrypting communications in Elasticsearch

發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

Elastic Certified Engineer複習記錄-集羣配置

死敌wen 2020-04-26 11:34:08

Elastic Certified Engineer複習記錄-集羣配置-題目篇

死敌wen 2020-04-26 11:34:08

Elastic Certified Engineer複習記錄-集羣配置-題目篇

死敌wen 2020-04-26 11:34:08