基於Mesos+Marathon+Zookeeper的Docker容器管理平臺

原創 tianpy5 2018-08-22 06:18

剛接觸docker的時候，發現docker很像一個虛擬機，隨着使用的增多，它給我們的開發工作帶來了極大的方便，因此就想自己搭建一個docker集羣環境，以便更深入瞭解docker集羣的工作原理。
之前曾經嘗試分別安裝mesos，marathon和zookeeper，組成分佈式系統，但是在安裝、配置過程中遇到了各種各樣的問題，特別是mesos，問題比較多。既然docker容器的使用十分方便，所以我就使用基於容器的方式來搭建docker集羣環境，我也推薦大家這樣做，特別方便。

1. 準備工作

1.1 三臺虛擬機

  • 192.168.56.103：ubuntu-1
  • 192.168.56.101：ubuntu-2
  • 192.168.56.102：ubuntu-3

1.2 應用部署

三臺主機分別部署zookeeper，mesos-master，mesos-slave，marathon

hostname application
ubuntu-1 zookeeper,mesos-master,mesos-slave,marathon
ubuntu-2 zookeeper,mesos-master,mesos-slave,marathon
ubuntu-3 zookeeper,mesos-master,mesos-slave,marathon

1.3 安裝docker環境

參考docker官網ubuntu操作系統安裝過程
https://docs.docker.com/engine/installation/linux/ubuntulinux/

2. 下載docker鏡像

docker pull mesoscloud/zookeeper   #zookeeper
docker pull mesoscloud/mesos-master   #mesos-master
docker pull mesoscloud/mesos-slave   #mesos-slave 
docker pull mesoscloud/marathon   #marathon 
docker pull davidcaste/alpine-tomcat   #tomcat

3. 啓動docker鏡像

以下命令分別在三臺主機運行，將ip地址和相關參數修改爲對應主機的ip地址和參數

3.1 運行zookeeper

docker run -d \
-e MYID=1 \ #每臺主機對應一個id，三臺主機分別對應1,2,3
-e SERVERS=192.168.56.103,192.168.56.101,192.168.56.102 \ #三臺主機的ip地址
--name=zookeeper \
--net=host \
--restart=always \
mesoscloud/zookeeper

3.2 運行mesos-master

docker run -d \
-e MESOS_HOSTNAME=192.168.56.103 \ #第一臺主機ip地址
-e MESOS_IP=192.168.56.103 \ #第一臺主機ip地址
-e MESOS_QUORUM=2 \
-e MESOS_ZK=zk://192.168.56.103:2181,192.168.56.101:2181,192.168.56.102:2181/mesos \
--name mesos-master \
--net host \
--restart=always \
mesoscloud/mesos-master

3.3 運行mesos-slave

docker run -d \
-e MESOS_HOSTNAME=192.168.56.103 \ #第一臺主機ip地址
-e MESOS_IP=192.168.56.103 \ #第一臺主機ip地址
-e MESOS_MASTER=zk://192.168.56.103:2181,192.168.56.101:2181,192.168.56.102:2181/mesos \
-v /sys/fs/cgroup:/sys/fs/cgroup \
-v /var/run/docker.sock:/var/run/docker.sock \
--name mesos-slave \
--net host \
--privileged \
--restart=always \
mesoscloud/mesos-slave

3.4 運行marathon

docker run -d \
-e MARATHON_HOSTNAME=192.168.56.103 \ #第一臺主機ip地址
-e MARATHON_HTTPS_ADDRESS=192.168.56.103 \ #第一臺主機ip地址
-e MARATHON_HTTP_ADDRESS=192.168.56.103 \ #第一臺主機ip地址
-e MARATHON_MASTER=zk://192.168.56.103:2181,192.168.56.101:2181,192.168.56.102:2181/mesos \
-e MARATHON_ZK=zk://192.168.56.103:2181,192.168.56.101:2181,192.168.56.102:2181/marathon \
--name marathon \
--net host \
--restart=always \
mesoscloud/marathon

4. 通過Marathon發佈應用

4.1 在第一臺一臺主機訪問http://192.168.56.103:8080，任何一臺主機的8080端口都可以訪問marathon，打開marathon UI，創建應用，點擊Create Application，填寫應用信息，發佈應用，Json模式數據如下：

{
  "id": "tomcat",
  "cmd": "/opt/tomcat/bin/catalina.sh run",
  "cpus": 1,
  "mem": 512,
  "disk": 512,
  "instances": 1,
  "container": {
    "docker": {
      "image": "davidcaste/alpine-tomcat",
      "network": "BRIDGE",
      "portMappings": [
        {
          "containerPort": 8080,
          "protocol": "tcp",
          "name": null
        }
      ],
      "parameters": []
    },
    "type": "DOCKER",
    "volumes": [
      {
        "hostPath": "/var/dockertest/logs",
        "containerPath": "/logs",
        "mode": "RW"
      }
    ]
  },
  "env": {},
  "labels": {},
  "healthChecks": []
}

4.2 也可以通過marathon提供的rest接口發佈應用

curl -X POST -H "Accept: application/json" -H "Content-Type: application/json" 192.168.56.103:8080/v2/apps -d'
{
  "id": "tomcat",
  "cmd": "/opt/tomcat/bin/catalina.sh run",
  "cpus": 1,
  "mem": 512,
  "disk": 512,
  "instances": 1,
  "container": {
    "docker": {
      "image": "davidcaste/alpine-tomcat",
      "network": "BRIDGE",
      "portMappings": [
        {
          "containerPort": 8080,
          "protocol": "tcp",
          "name": null
        }
      ],
      "parameters": []
    },
    "type": "DOCKER",
    "volumes": [
      {
        "hostPath": "/var/dockertest/logs",
        "containerPath": "/logs",
        "mode": "RW"
      }
    ]
  },
  "env": {},
  "labels": {},
  "healthChecks": []
}'

5. 查看應用狀態

5.1 marathon主頁面（http://192.168.56.103:8080）

Marathon主頁面

5.2 擊進入應用頁面，只有一個實例

進入應用頁面

5.3 點擊Scale Application擴展三個實例

擴展三個實例

5.4 打開mesos主頁面（http://192.168.56.103:5050），可以看到有三個tomcat實例

mesos主頁面

5.5 在Mesos中打開Framework頁面，我們可以看到marathon框架的正在運行。

Mesos-Framework頁面

5.6 接下來我們看一下mesos-slave頁面，看到運行正常的三個mesos-slave。

mesos-slave頁面

6. 調用服務

我們通過marathon把tomcat運行起來了，但是tomcat裏沒有war包，所以我們將準備好的一個war包放進三臺主機的tomcat裏，這裏是爲了測試方便，我們直接使用docker cp命令將war包複製到docker容器當中。在實際應用中我們可以通過其他方式來加載war包，比如，在CMD命令裏通過wget方式將war包下載到容器裏，或者通過卷Volume的方式共享，將war包放到tomcat的webaps目錄下。

suser@ubuntu-1:~$ docker ps
CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS              PORTS                     NAMES
dd014c78d8ec        davidcaste/alpine-tomcat   "/bin/sh -c '/opt/tom"   17 minutes ago      Up 17 minutes       0.0.0.0:31204->8080/tcp   mesos-81974b67-e403-47a0-bffd-7cbfa03da51b-S0.e9647bd1-2771-487b-a01e-6a0ae68c62f9
601dbe2256d5        mesoscloud/marathon        "/usr/local/bin/dumb-"   5 hours ago         Up 5 hours                                    marathon
1dfaabf2cd3e        mesoscloud/mesos-slave     "/usr/local/bin/dumb-"   5 hours ago         Up 5 hours                                    mesos-slave
e94c3202f529        mesoscloud/mesos-master    "/usr/local/bin/dumb-"   5 hours ago         Up 5 hours                                    mesos-master
a3389a386a11        mesoscloud/zookeeper       "/usr/local/bin/dumb-"   5 hours ago         Up 5 hours                                    zookeeper
suser@ubuntu-1:~$ docker cp /mnt/share/backendresource.war dd014c78d8ec:/opt/tomcat/webapps #dd014c78d8ec爲Container ID
suser@ubuntu-1:~$

接下來我們訪問服務查看返回結果

suser@ubuntu-1:~$ curl 192.168.56.102:31515/backendresource/11/1/1
{"ACT_TAG":"1","X_RESULTCODE":"0"}
suser@ubuntu-1:~$ curl 192.168.56.101:31299/backendresource/11/1/1
{"ACT_TAG":"1","X_RESULTCODE":"0"}
suser@ubuntu-1:~$ curl 192.168.56.103:31204/backendresource/11/1/1
{"ACT_TAG":"1","X_RESULTCODE":"0"}

查看docker容器日誌

suser@ubuntu-1:~$ docker logs dd014c78d8ec
log4j:WARN No such property [encoding] in org.apache.log4j.net.SyslogAppender.
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2016-10-15 21:46:27 [INFO ](BackendResources              ) [TxId :  , SpanId : ] 榛戝悕鍗昩ackendresource緇撴灉:{"ACT_TAG":"1","X_RESULTCODE":"0"},鏈璋冪敤鑰楁椂:1117ms
2016-10-15 21:47:13 [INFO ](BackendResources              ) [TxId :  , SpanId : ] 榛戝悕鍗昩ackendresource緇撴灉:{"ACT_TAG":"0","X_RESULTCODE":"0"},鏈璋冪敤鑰楁椂:2ms
suser@ubuntu-1:~$

在容器也看到了調用日誌，看到中文是亂碼，這個可能是容器字符編碼問題。

7. 通過marathon rest接口獲取數據

#通過marathon接口獲取所有應用信息
suser@ubuntu-1:~$ curl 192.168.56.103:8080/v2/apps
{"apps":[{"id":"/tomcat","cmd":"/opt/tomcat/bin/catalina.sh run","args":null,"user":null,"env":{},"instances":3,"cpus":0.5,"mem":512,"disk":512,"executor":"","constraints":[],"uris":[],"fetch":[],"storeUrls":[],"ports":[10000],"portDefinitions":[{"port":10000,"protocol":"tcp","labels":{}}],"requirePorts":false,"backoffSeconds":1,"backoffFactor":1.15,"maxLaunchDelaySeconds":3600,"container":{"type":"DOCKER","volumes":[],"docker":{"image":"davidcaste/alpine-tomcat","network":"BRIDGE","portMappings":[{"containerPort":8080,"hostPort":0,"servicePort":10000,"protocol":"tcp","labels":{}}],"privileged":false,"parameters":[],"forcePullImage":false}},"healthChecks":[],"readinessChecks":[],"dependencies":[],"upgradeStrategy":{"minimumHealthCapacity":1,"maximumOverCapacity":1},"labels":{},"acceptedResourceRoles":null,"ipAddress":null,"version":"2016-10-15T21:20:55.434Z","residency":null,"versionInfo":{"lastScalingAt":"2016-10-15T21:20:55.434Z","lastConfigChangeAt":"2016-10-15T16:58:42.119Z"},"tasksStaged":0,"tasksRunning":3,"tasksHealthy":0,"tasksUnhealthy":0,"deployments":[]}]}suser@ubuntu-1:~$ 

##通過marathon接口獲取tomcat應用信息
suser@ubuntu-1:~$ curl 192.168.56.103:8080/v2/apps/tomcat
{"app":{"id":"/tomcat","cmd":"/opt/tomcat/bin/catalina.sh run","args":null,"user":null,"env":{},"instances":3,"cpus":0.5,"mem":512,"disk":512,"executor":"","constraints":[],"uris":[],"fetch":[],"storeUrls":[],"ports":[10000],"portDefinitions":[{"port":10000,"protocol":"tcp","labels":{}}],"requirePorts":false,"backoffSeconds":1,"backoffFactor":1.15,"maxLaunchDelaySeconds":3600,"container":{"type":"DOCKER","volumes":[],"docker":{"image":"davidcaste/alpine-tomcat","network":"BRIDGE","portMappings":[{"containerPort":8080,"hostPort":0,"servicePort":10000,"protocol":"tcp","labels":{}}],"privileged":false,"parameters":[],"forcePullImage":false}},"healthChecks":[],"readinessChecks":[],"dependencies":[],"upgradeStrategy":{"minimumHealthCapacity":1,"maximumOverCapacity":1},"labels":{},"acceptedResourceRoles":null,"ipAddress":null,"version":"2016-10-15T21:20:55.434Z","residency":null,"versionInfo":{"lastScalingAt":"2016-10-15T21:20:55.434Z","lastConfigChangeAt":"2016-10-15T16:58:42.119Z"},"tasksStaged":0,"tasksRunning":3,"tasksHealthy":0,"tasksUnhealthy":0,"deployments":[],"tasks":[{"id":"tomcat.a143d29d-92f8-11e6-b002-9e1a99d79ec2","slaveId":"81974b67-e403-47a0-bffd-7cbfa03da51b-S2","host":"192.168.56.102","startedAt":"2016-10-15T16:58:48.013Z","stagedAt":"2016-10-15T16:58:43.876Z","ports":[31515],"version":"2016-10-15T16:58:42.119Z","ipAddresses":[{"ipAddress":"172.17.0.2","protocol":"IPv4"}],"appId":"/tomcat"},{"id":"tomcat.424d101e-931d-11e6-b002-9e1a99d79ec2","slaveId":"81974b67-e403-47a0-bffd-7cbfa03da51b-S1","host":"192.168.56.101","startedAt":"2016-10-15T21:21:06.234Z","stagedAt":"2016-10-15T21:20:55.709Z","ports":[31299],"version":"2016-10-15T21:20:55.434Z","ipAddresses":[{"ipAddress":"172.17.0.2","protocol":"IPv4"}],"appId":"/tomcat"},{"id":"tomcat.4263a55f-931d-11e6-b002-9e1a99d79ec2","slaveId":"81974b67-e403-47a0-bffd-7cbfa03da51b-S0","host":"192.168.56.103","startedAt":"2016-10-15T21:21:10.236Z","stagedAt":"2016-10-15T21:20:55.858Z","ports":[31204],"version":"2016-10-15T21:20:55.434Z","ipAddresses":[{"ipAddress":"172.17.0.2","protocol":"IPv4"}],"appId":"/tomcat"}]}}suser@ubuntu-1:~$

8. 總結

通過以上過程大家可以發現直接把docker容器起來就應用可以使用了，通過容器來運行服務簡單而高效，省去了很多不必要的麻煩。相信隨着越來越多的人使用docker技術，不斷的推動docker技術的快速發展，docker技術會更加流行，應用會更加廣泛。

發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

linux上安裝Docker(非常簡單的安裝方法)

幸運券發放 2019-02-24 19:38:01

Error response from daemon: OCI runtime create failed

uiz 2019-02-24 13:44:10

『高級篇』docker之kubernetes基礎集羣附加功能kube-proxy和kube-dns（

IT人故事 2019-02-23 14:07:34

寶塔面板 + Rancher + 阿里雲鏡像倉庫 + Docker + Kubernetes，添加集羣、部署 web 應用

燕小范 2019-02-23 13:20:14

jenkins觸發式自動構建docker鏡像上傳至harbor併發布

龍澤 2019-02-23 13:07:50

Flask 教程 第十九章：Docker容器上的部署

天降攻城獅 2019-02-23 10:17:16

docker

xmlgrg 2019-02-23 00:41:20

docker 基礎命令

qq5a3c5e337fedf 2019-02-23 00:40:07

docker容器初探—基本概念和基礎命令用法

IT陳工 2019-02-23 00:35:57

ZooKeeper实战与源码剖析

wy53780 2020-04-23 15:39:23

Zookeeper安裝部署

lvrenX 2019-02-24 13:04:33

zookeeper環境搭建中的幾個坑[Error contacting service. It is probably not running]的分析及解決

vfast_wuwei 2019-02-23 12:55:35

Curator: ZooKeeper的使用配方

Yujiaao 2019-02-23 01:52:43

zookeeper kafka集羣安裝配置

sqtce 2019-02-23 00:42:02

hadoop搭建之hive安裝

伊人心 2019-02-23 00:41:25