Docker之 docker 17.03-ce 模擬nginx負載均衡

環境：

OS X ，vagrant 1.9 [ centos 7.3 ] 

安裝docker-ce：

[vagrant@vgt-centos7-docker1 ~]$ sudo yum install -y yum-utils device-mapper-persistent-data lvm2
......

[vagrant@vgt-centos7-docker1 ~]$ sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
......
[vagrant@vgt-centos7-docker1 ~]$ sudo yum makecache fast
......
元數據緩存已建立

[vagrant@vgt-centos7-docker1 ~]$ yum list docker-ce.x86_64  --showduplicates |sort -r
已加載插件：fastestmirror
可安裝的軟件包
 * updates: mirror.bit.edu.cn
 * extras: mirror.bit.edu.cn
docker-ce.x86_64            17.03.1.ce-1.el7.centos             docker-ce-stable
docker-ce.x86_64            17.03.0.ce-1.el7.centos             docker-ce-stable
Determining fastest mirrors
 * base: mirror.bit.edu.cn

[vagrant@vgt-centos7-docker1 ~]$ sudo yum install docker-ce-17.03.1.ce
......
[vagrant@vgt-centos7-docker1 ~]$ docker version
Client:
Version:      17.03.1-ce
......
[vagrant@vgt-centos7-docker1 ~]$ sudo systemctl start docker
......

安裝 docker-compose： 

兩種方式：

1、pip 安裝 （推薦）

1）、安裝 pip

2) 、使用 pip 安裝 docker-compose

[vagrant@vgt-centos7-docker1 lb]$ sudo pip install -U docker-compose
...
[vagrant@vgt-centos7-docker1 lb]$ docker-compose -version
docker-compose version 1.14.0, build c7bdf9e

#添加 bash 補全命令（直接瀏覽器先測試下對應版本，如 1.14.0 是否存在）
[root@vgt-centos7-docker1 ~]# curl -L https://raw.githubusercontent.com/docker/compose/1.14.0/contrib/completion/bash/docker-compose > /etc/bash_completion.d/docker-compose

2、二進制

發佈的二進制包及安裝方法，可以在 https://github.com/docker/compose/releases 找到。

準備測試目錄及終端環境

# 可以先 echo $TERM 如果是不是xterm 可使用如下命令修改
[vagrant@vgt-centos7-docker1 lb]$ echo 'export TERM=xterm' >> ~/.bashrc
....

[vagrant@vgt-centos7-docker1 ~]$ sudo mkdir -p /home/vagrant/docker/lb
......
[vagrant@vgt-centos7-docker1 ~]$ cd /home/vagrant/docker/lb
...

xterm 是 TERM 的一種終端類型，除了xterm類型外，還有 vt100 等其他類型，區別解釋如下：

xterm is supposed to be a superset of vt220, in other words it's like vt220 but has more features. For example, xterm usually supports colors, but vt220 doesn't. You can test this by pressing z inside top.

In the same way, vt220 has more features than vt100. For example, vt100 doesn't seem to support F11 and F12.

容器編排負載均衡集羣

1. 目錄如下：

   [vagrant@vgt-centos7-docker1 lb]$ tree

   .

   ├── conf

   │   ├── nginx01.conf

   │   ├── nginx02.conf

   │   └── nginx.conf

   └── docker-compose.yml

2. 各個文件內容詳解：

   # FILE: docker-compose.yml
   # version 號參考 https://github.com/docker/compose/releases/ 中的 “ Compose file format” ，不同版本有不同的參數
   
   version: '3'
   services:
           mysql:
             image: hub.c.163.com/library/mysql:5.6
             volumes:
               - "/data/mysql:/var/lib/mysql"
             #使用 HOST : CONTAINER 格式，或者只是指定容器的端口，宿主機會隨機映射端口
             ports:
               - "3306:3306"
             environment:
               - MYSQL_ROOT_PASSWORD=123456
             restart: always
   
           # 服務名稱phpfpm，自定義
           phpfpm01:
             links:
               - mysql
             image: hub.c.163.com/library/php:5.6.30-fpm
             volumes:
               - "/home/wwwroot/default:/home/wwwroot"
             ports:
               - "9001:9000"
             restart: always
   
           phpfpm02:
             links:
               - mysql
             image: hub.c.163.com/library/php:5.6.30-fpm
             volumes:
               - "/home/wwwroot/default:/home/wwwroot"
             ports:
               - "9002:9000"
             restart: always
   
           # 在 nginx 上反向代理作 nginx ,nginx01,nginx02 三者的簡單負載，同時根據不同負載到的 nginx 再均衡到 phpfpm01 
             和 phpfpm02
   
           nginx:
             image: oliverlin/centos67_nginx:1.9.5
             links:
               - nginx01
               - nginx02
               - phpfpm01
               - phpfpm02
             volumes:
               - "./conf/nginx.conf:/usr/local/nginx/conf/vhost/docker1.dev.cc.conf"
             ports:
               - "80:80"
               - "443:443"
             restart: always
   
           nginx01:
             links:
               - phpfpm01
               - phpfpm02
             image: oliverlin/centos67_nginx:1.9.5
             volumes:
               - "./conf/nginx01.conf:/usr/local/nginx/conf/vhost/docker1.dev.cc.conf"
             ports:
               - "11080:80"
               - "11443:443"
             restart: always
   
           nginx02:
             image: oliverlin/centos67_nginx:1.9.5
             links:
               - phpfpm01
               - phpfpm02
             volumes:
               - "./conf/nginx02.conf:/usr/local/nginx/conf/vhost/docker1.dev.cc.conf"
             ports:
               - "12080:80"
               - "12443:443"
             restart: always
   
   

   # conf/nginx.conf
   
   upstream ngxproxy {
           server localhost:10080 weight=4;
           server lb_nginx01_1:80 weight=5;
           server lb_nginx02_1:80 weight=3;
   }
   
   upstream fpmproxy{
       server lb_phpfpm01_1:9000;
       server lb_phpfpm02_1:9000;
   }
   
   server {
           listen      80;
           server_name docker1.dev.cc;
           index index.html index.htm index.php;
           access_log /var/log/wwwlogs/access_nginx.log combined;
           location / {
                   # 反向代理的主機頭
                   proxy_pass  http://ngxproxy;
                   proxy_set_header Host   $host;
                   proxy_set_header   X-Real-IP        $remote_addr;
                   proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
           }
   }
   
   server {
           listen 10080;
           server_name localhost;
           index index.html index.php index.jsp;
           access_log /var/log/wwwlogs/access_nginx.log combined;
   
           location ~ \\.(php|php5)?$ {
                   root            /home/wwwroot;
                   #fastcgi_pass    dckdev_phpfpm_1:9000;
                   fastcgi_pass    fpmproxy;
                   fastcgi_index   index.php;
                   fastcgi_param   SCRIPT_FILENAME /home/wwwroot$fastcgi_script_name;
                   include         fastcgi_params;
           }
   
   
           location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|flv|ico)$ {
                   expires 30d;
           }
   }
   

   # conf/nginx01.conf
   server {
           # 容器與容器間的端口訪問 都用80 即可
           listen 80;
           server_name docker1.dev.cc;
           index index.html index.php;
           access_log /var/log/wwwlogs/access_nginx.log combined;
   
           location /status {
                   stub_status on;
                   auth_basic "WebServer Status";
           }
   
           location ~ \\.(php|php5)?$ {
                   root            /home/wwwroot;
                   # docker-compose 編排後生成的容器 name 爲 lb(項目名，默認文件夾名)_phpfpm01(compose文件中定義的link的容器名)_1(表示第一組)
                   fastcgi_pass    lb_phpfpm01_1:9000;
                   fastcgi_index   index.php;
                   fastcgi_param   SCRIPT_FILENAME $document_root$fastcgi_script_name;
                   include         fastcgi_params;
           }
   
           location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|flv|ico)$ {
                   expires 30d;
           }
   }
   

   # conf/nginx02.conf
   server {
           listen 80;
           server_name docker1.dev.cc;
           index index.html index.php index.jsp;
           access_log /var/log/wwwlogs/access_nginx.log combined;
   
           location /status {
                   stub_status on;
                   auth_basic "WebServer Status";
           }
   
           location ~ \\.(php|php5)?$ {
                   root            /home/wwwroot;
                   fastcgi_pass    lb_phpfpm02_1:9000;
                   fastcgi_index   index.php;
                   fastcgi_param   SCRIPT_FILENAME $document_root$fastcgi_script_name;
                   include         fastcgi_params;
           }
   
           location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|flv|ico)$ {
                   expires 30d;
           }
   }
   

    

3. 最終效果：

   3.1 容器列表：
   
   
   lb_nginx_1 ： 127.0.0.1 （入口）
   lb_nginx01_1：172.18.0.6
   lb_nginx02_1：172.18.0.5
   
   container name    container ID
   lb_phpfpm01_1 ：a24d273739e4                              
   lb_phpfpm02_1： c4bdd1f07fe4                         
   
   3.2 不斷刷新瀏覽器，顯示效果：
   
   
   
   
   

遇到的問題 

1. 超時

ERROR: for repository_nginx_1  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

ERROR: for nginx  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

解決方法：

## 如果只有 docker-compose.yml 則可以不用 -f *.yml 參數
[root@vgt-centos7-docker1 ]# COMPOSE_HTTP_TIMEOUT=200 docker-compose -f docker-compose-test1.yml up

2. 

[root@localhost ~]# yum install docker-ce-17.03.1.ce-1.el7.centos 
已加載插件：fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.yun-idc.com
 * extras: mirrors.163.com
 * updates: mirrors.btte.net
正在解決依賴關係
--> 正在檢查事務
---> 軟件包 docker-ce.x86_64.0.17.03.1.ce-1.el7.centos 將被 安裝
--> 正在處理依賴關係 docker-ce-selinux >= 17.03.1.ce-1.el7.centos，它被軟件包 docker-ce-17.03.1.ce-1.el7.centos.x86_64 需要
軟件包 docker-ce-selinux 已經被 docker-ce 取代，但是取代的軟件包並未滿足需求
--> 解決依賴關係完成
錯誤：軟件包：docker-ce-17.03.1.ce-1.el7.centos.x86_64 (docker-ce-stable)
          需要：docker-ce-selinux >= 17.03.1.ce-1.el7.centos
          可用: docker-ce-selinux-17.03.0.ce-1.el7.centos.noarch (docker-ce-stable)
              docker-ce-selinux = 17.03.0.ce-1.el7.centos
          可用: docker-ce-selinux-17.03.1.ce-1.el7.centos.noarch (docker-ce-stable)
              docker-ce-selinux = 17.03.1.ce-1.el7.centos
          可用: docker-ce-selinux-17.03.2.ce-1.el7.centos.noarch (docker-ce-stable)
              docker-ce-selinux = 17.03.2.ce-1.el7.centos
 您可以嘗試添加 --skip-broken 選項來解決該問題
** 發現 4 個已存在的 RPM 數據庫問題， 'yum check' 輸出如下：
ipa-client-4.4.0-14.el7.centos.7.x86_64 有已安裝衝突 freeipa-client: ipa-client-4.4.0-14.el7.centos.7.x86_64
ipa-client-common-4.4.0-14.el7.centos.7.noarch 有已安裝衝突 freeipa-client-common: ipa-client-common-4.4.0-14.el7.centos.7.noarch
ipa-common-4.4.0-14.el7.centos.7.noarch 有已安裝衝突 freeipa-common: ipa-common-4.4.0-14.el7.centos.7.noarch
ipa-python-compat-4.4.0-14.el7.centos.7.noarch 有已安裝衝突 freeipa-python-compat: ipa-python-compat-4.4.0-14.el7.centos.7.noarch

解決方法：

A new obsoletes restriction was introduced in docker-ce 17.06.0 and it looks like the yum repo is applying it to all versions of the docker-ce indiscriminately. To work around this when installing older versions of docker-ce, you can pass a flag to ignore obsoletes:

$ yum install -y --setopt=obsoletes=0 \
  docker-ce-17.03.1.ce-1.el7.centos \
  docker-ce-selinux-17.03.1.ce-1.el7.centos

其他