一、簡述
（一）基礎環境
- 四個節點：192.168.1.171~174。
- 操作系統：centos7。
- docker版本：19.03.8。
（二）目標
- 無狀態服務多實例伸縮部署。
- 掛載存儲多實例部署。
- 對集羣外開放訪問端口。
- 定義集羣網絡，以服務名訪問服務。
- 多manager節點。
二、部署
（一）部署準備
更改主機名
swarm使用主機名來展示節點，更改主機名爲有意義的名字，這裏改爲test171~test174。
[root@localhost ~]# hostnamectl set-hostname test171
[root@test171 ~]#
修改/etc/hosts文件
添加：
192.168.1.171 test1
192.168.1.172 test2
192.168.1.173 test3
192.168.1.174 test4
打開必須端口
#管理端口
firewall-cmd --zone=public --add-port=2377/tcp --permanent
#節點間通信端口
firewall-cmd --zone=public --add-port=7946/tcp --permanent
firewall-cmd --zone=public --add-port=7946/udp --permanent
#overlay 網絡端口
firewall-cmd --zone=public --add-port=4789/tcp --permanent
firewall-cmd --zone=public --add-port=4789/udp --permanent
#起效
firewall-cmd --reload
安裝docker
不詳述。
（二）部署swarm
創建集羣
在test171上創建集羣，test171將作爲manager節點。
[root@test171 ~]# docker swarm init --advertise-addr 192.168.1.171
Swarm initialized: current node (jp9mooyi4go7ytibxbcdevmmj) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join --token SWMTKN-1-59w2aklummvra2w6cjjjagssoywqbgmnpc2qq01389bmhf2l8x-abh90k1sojtpgvsdolp52k88n 192.168.1.171:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
節點加入集羣
在其他節點上執行上面的提示，加入集羣
docker swarm join --token SWMTKN-1-59w2aklummvra2w6cjjjagssoywqbgmnpc2qq01389bmhf2l8x-abh90k1sojtpgvsdolp52k88n 192.168.1.171:2377
在test171這個manager節點上：
執行docker node ls可以查看集羣節點
[root@test171 data]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
jp9mooyi4go7ytibxbcdevmmj * test171 Ready Active Leader 19.03.8
0zovd07bxee4cvk9papukatnm test172 Ready Active 19.03.8
i1ep20qu560v2omuxpi8x2xe8 test173 Ready Active 19.03.8
zww4t1lebp2dtftareudqpx53 test174 Ready Active 19.03.8
執行docker swarm join-token -q worker可以打印集羣token
[root@test171 data]# docker swarm join-token -q worker
SWMTKN-1-59w2aklummvra2w6cjjjagssoywqbgmnpc2qq01389bmhf2l8x-abh90k1sojtpgvsdolp52k88n
三、使用
（一） 創建overlay network網絡
使用overlay network網絡。overlay network的最實用的好處，一是在整個swarm集羣內實現跨主機通信；而是可以通過service名稱進行訪問而不用指定容器ip地址。在manager節點上執行
[root@test171 data]# docker network create --driver overlay --opt encrypted --subnet 192.168.84.0/24 test_net
reu4c2euepogf7cux08cij3bw
[root@test171 data]# docker network ls
NETWORK ID NAME DRIVER SCOPE
73b7f9e4b2f4 bridge bridge local
cdc88f3bec17 docker_gwbridge bridge local
3f2652b5241e host host local
8166p62br08o ingress overlay swarm
d1135a68eb99 none null local
reu4c2euepog test_net overlay swarm
- –opt encrypted：通信加密
- –subnet xxx：指定IP地址段
- test_net：指定的網絡名稱
（二）創建本地倉庫
創建本地倉庫用於在各節點間同步鏡像文件
#給第一個節點打個標籤：key是func，值是registry
docker node update --label-add func=registry node1
#只在第一個節點上運行
docker service create --constraint 'node.labels.func == registry' --replicas 1 --network test_net --name registry --publish 5000:5000 registry:latest
訪問http://192.168.1.171:5000/v2/_catalog，在四個節點的ip地址上任意一個都可以訪問這個倉庫。
修改/etc/docker/daemon.json，添加"insecure-registries": [“192.168.1.171:5000”]
（三）創建無狀態服務的多實例
在manager節點上執行。創建一個服務，命名爲testdemo，從本地的倉庫里拉取一個鏡像，共起五個實例。這個鏡像提供restful api訪問服務，訪問8888端口。
創建服務
[root@test171 data]# docker service create --replicas 5 --network test_net --name testdemo 192.168.1.164:8001/dev/demo:build
image 192.168.1.164:8001/dev/demo:build could not be accessed on a registry to record
its digest. Each node will access 192.168.1.164:8001/dev/demo:build independently,
possibly leading to different nodes running different
versions of the image.
q2x00zgi5tzu5ieaftga879hh
overall progress: 5 out of 5 tasks
1/5: running [==================================================>]
2/5: running [==================================================>]
3/5: running [==================================================>]
4/5: running [==================================================>]
5/5: running [==================================================>]
verify: Service converged
查看服務狀態
[root@test171 data]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
q2x00zgi5tzu testdemo replicated 5/5 192.168.1.164:8001/dev/demo:buil
[root@test171 data]# docker service ps testdemo
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
r2rbujc4ydhe testdemo.1 192.168.1.164:8001/dev/demo:build test174 Running Running 42 seconds ago
kzjr70sqquio testdemo.2 192.168.1.164:8001/dev/demo:build test171 Running Running 43 seconds ago
3zh2ets41dp9 testdemo.3 192.168.1.164:8001/dev/demo:build test172 Running Running 43 seconds ago
ef9zn7v4j0nq testdemo.4 192.168.1.164:8001/dev/demo:build test172 Running Running 43 seconds ago
s2anzuf09g3b testdemo.5 192.168.1.164:8001/dev/demo:build test173 Running Running 37 seconds ago
ubqfejj0i235 \_ testdemo.5 192.168.1.164:8001/dev/demo:build test173 Shutdown Failed 43 seconds ago "starting container failed: fa…"
可以看到test172上起了兩個實例，一共5個實例。
查看網絡訪問
[root@test171 data]# docker network inspect test_net
[
{
"Name": "test_net",
"Id": "reu4c2euepogf7cux08cij3bw",
"Created": "2020-03-24T16:29:04.51653612+08:00",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "192.168.84.0/24",
"Gateway": "192.168.84.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"604da0371c0a21404778f65ea1791219333d88ae070ee0edd5145e6404cff7f2": {
"Name": "testdemo.2.kzjr70sqquiodpiyaadaxf52q",
"EndpointID": "4ec2a9ba55585233373f01c660b06881dc9e6759065c75fc9ec2404ec8290c9a",
"MacAddress": "02:42:c0:a8:54:87",
"IPv4Address": "192.168.84.135/24",
"IPv6Address": ""
},
"lb-test_net": {
"Name": "test_net-endpoint",
"EndpointID": "6d3b7a6d28b52d6261b7384732dfd553255bb7c610fc4ab25d88302eaf4047ba",
"MacAddress": "02:42:c0:a8:54:7d",
"IPv4Address": "192.168.84.125/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4097",
"encrypted": ""
},
"Labels": {},
"Peers": [
{
"Name": "43be15a3ab8b",
"IP": "192.168.1.171"
}
]
}
]
動態調整實例數目
由5個實例調整爲2個。
[root@test171 data]# docker service ps testdemo
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
r2rbujc4ydhe testdemo.1 192.168.1.164:8001/dev/demo:build test174 Running Running 11 minutes ago
kzjr70sqquio testdemo.2 192.168.1.164:8001/dev/demo:build test171 Running Running 11 minutes ago
3zh2ets41dp9 testdemo.3 192.168.1.164:8001/dev/demo:build test172 Running Running 11 minutes ago
ef9zn7v4j0nq testdemo.4 192.168.1.164:8001/dev/demo:build test172 Running Running 11 minutes ago
s2anzuf09g3b testdemo.5 192.168.1.164:8001/dev/demo:build test173 Running Running 11 minutes ago
ubqfejj0i235 \_ testdemo.5 192.168.1.164:8001/dev/demo:build test173 Shutdown Failed 11 minutes ago "starting container failed: fa…"
[root@test171 data]# docker service scale testdemo=2
testdemo scaled to 2
overall progress: 2 out of 2 tasks
1/2: running [==================================================>]
2/2: starting container failed: failed to get network during CreateEndpoint: ne…
verify: Service converged
[root@test171 data]# docker service ps testdemo
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
r2rbujc4ydhe testdemo.1 192.168.1.164:8001/dev/demo:build test174 Running Running 12 minutes ago
kzjr70sqquio testdemo.2 192.168.1.164:8001/dev/demo:build test171 Running Running 12 minutes ago
ubqfejj0i235 testdemo.5 192.168.1.164:8001/dev/demo:build test173 Shutdown Failed 12 minutes ago "starting container failed: fa…"
（四）開放端口
創建服務時，加上–publish，即可將端口開放。如docker service create --replicas 1 --network test_net --name testnginx -p 80:80 nginx，在集羣內所有機器上都可以通過80端口訪問nginx，而實際上實例只有一個。
（五）綁定卷
創建服務時，使用--mount type=volume,src=testvolume,dst=/zjz或者--mount type=bind,target=/container_data/,source=/host_data/參數即可綁定卷或者主機目錄。
需要注意的是，每一個運行了服務容器的節點都將會開啓卷或者是綁定目錄，從而實現數據同步。
（六）容器內訪問
容器內訪問時可直接訪問服務名，在本例中科通過訪問testdemo來訪問提供的服務接口，實質上通過服務名獲取一個代理地址，代理掃描提供服務的容器來提供服務。
四、附錄
（一）常用命令
#列舉節點
docker node ls
#上線節點
docker node update --availability active test172
#下線節點
docker node update --availability drain test172
#刪除節點
docker node rm --force test172
#創建overlay網絡
docker network create --driver overlay --opt encrypted --subnet 192.168.84.0/24 test_net
#列舉網絡
docker network ls
#查看網絡詳情
docker network inspect test_net
#創建服務
docker service create --replicas 5 --network test_net --name testnginx --publish 80:80 nginx
#列舉服務
docker service ls
#查看服務狀態
docker service ps testdemo
docker service inspect --pretty testdemo
#動態調整服務實例數目
docker service scale my_nginx=1
#更新服務的鏡像
docker service update --image nginx:new my_nginx
#清除無效鏡像
docker rmi $(docker images -f "dangling=true" -q)