Docker registry V2 nginx 搭建
一 環境
倆臺centos 7 64位
Docker 版本
Client:
Version: 1.8.2
API version: 1.20
Package Version: docker-1.8.2-7.el7.centos.x86_64
Go version: go1.4.2
Git commit: bb472f0/1.8.2
Built:
OS/Arch: linux/amd64
Server:
Version: 1.8.2
API version: 1.20
Package Version:
Go version: go1.4.2
Git commit: bb472f0/1.8.2
Built:
OS/Arch: linux/amd64
Register V2.0
Register server 鏡像服務器 192.168.33.75
Register client 測試鏡像docker服務器 192.168.153.86
二 搭建過程
在Register server 上（192.168.33.75）
2.1 安裝需要的軟件
yum install gcc make pcre-devel pcre openssl-devel httpd-tools -y
其中 pcre-devel pcre 是在nginx需要的用的軟件包
Openssl-devel 是製作key的時候需要用的軟件包
Httpd-tools 是生成nginx用戶密碼需要用的軟件包
2.2 更改hosts,添加
[root@localhost ~]# vim /etc/hosts
192.168.33.75 dockertest.xxxx.com
2.3 生成根密鑰
先看下/etc/pki/CA 下有沒有
Cacert.pem index.txt index.txt.attr index.txt.old serial serial.old
有的話全部刪除，然後
[root@localhost ~]# cd /etc/pki/CA/
生成根密鑰
[root@localhost ~]#openssl genrsa -out private/cakey.pem 2048
生成根證書
openssl req -new -x509 -key private/cakey.pem -out cacert.pem
輸出，其中最主要的是紅色表明的
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:beijing
Locality Name (eg, city) [Default City]:beijing
Organization Name (eg, company) [Default Company Ltd]:beijing
Organizational Unit Name (eg, section) []:beijing
Common Name (eg, your name or your server's hostname) []:dockertest.xxxx.com
Email Address []:[email protected]
其中CN一定要寫自己的私有鏡像庫的域名
Email要記住，後面要用到
上面的自簽證書cacert.pem在/etc/pki/CA/目錄下
2.4 爲nginx web服務器生成ssl密鑰
[root@localhost CA]# mkdir /usr/local/nginx/ssl
[root@localhost CA]# cd /usr/local/nginx/ssl
[root@localhost CA]# openssl genrsa -out nginx.key 2048
爲nginx生成證書籤署請求
[root@localhost CA]# openssl req -new -key nginx.key -out nginx.csr
輸出，其中最主要的是紅色表明的
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:beijing
Locality Name (eg, city) [Default City]:beijing
Organization Name (eg, company) [Default Company Ltd]:beijing
Organizational Unit Name (eg, section) []:beijing
Common Name (eg, your name or your server's hostname) []:dockertest.xxxx.com
Email Address []:[email protected]
A challenge password []:
An optional company name []:
其中Common Name 和email要和上面的一樣，challenge password不填
私有CA根據請求來簽發證書
[root@localhost CA]# touch /etc/pki/CA/index.txt
[root@localhost CA]# touch /etc/pki/CA/serial
[root@localhost CA]# echo 00 > /etc/pki/CA/serial
[root@localhost CA]#openssl ca -in nginx.csr -out nginx.crt
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 1 (0x1)
Validity
Not Before: Jan 11 06:04:22 2016 GMT
Not After : Jan 10 06:04:22 2017 GMT
Subject:
countryName = CN
stateOrProvinceName = beijing
organizationName = beijing
organizationalUnitName = beijing
commonName = dockertest.hc360.org
emailAddress = [email protected]
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
80:2C:A4:5D:55:92:2E:E0:38:26:9A:A0:F2:7E:29:6A:24:3D:FE:DF
X509v3 Authority Key Identifier:
keyid:60:BE:EA:F3:43:79:DC:46:D3:A2:00:2A:2E:8A:D6:10:93:5C:54:59
Certificate is to be certified until Jan 10 06:04:22 2017 GMT (365 days)
Sign the certificate? [y/n]:y
直接輸入y，確認就好了。
三 安裝配置nginx
3.1 安裝nginx
Nginx 版本必須大於1.7.5，我使用的是1.8的
[root@localhost CA]# tar zxvf nginx-1.8.0.tar.gz
cd nginx-1.8.0
mkdir /usr/local/nginx
./configure --prefix=/usr/local/nginx --with-http_ssl_module --with-http_stub_status_module --with-pcre --with-http_addition_module --with-http_realip_module --with-http_flv_module
make && make install
3.2 生成htpasswd
[root@localhost]# htpasswd -cb /usr/local/nginx/conf/.htpasswd admin admin
生成的用戶名和密碼都是admin，到時登錄私有倉庫的時候用。
3.3 配置nginx
[root@localhost]# cd /usr/local/nginx/conf/
[root@localhost]# cp nginx.conf nginx.conf.bak
[root@localhost]# vim nginx.conf
user root root;
worker_processes auto;
error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
pid logs/nginx.pid;
worker_rlimit_nofile 51200;
events {
use epoll;
worker_connections 51200;
multi_accept on;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
server_names_hash_bucket_size 128;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
upstream registry {
server 127.0.0.1:5000;
}
server {
listen 443;
server_name 192.168.33.75;
add_header Docker-Distribution-Api-Version registry/2.0 always;
sslon;
ssl_certificate /usr/local/nginx/ssl/nginx.crt; #crt的位置
ssl_certificate_key /usr/local/nginx/ssl/nginx.key; #key的位置
client_max_body_size 0;
chunked_transfer_encoding on;
location / {
auth_basic "registry";
auth_basic_user_file /usr/local/nginx/conf/.htpasswd; #htpasswd的位置
root html;
index index.html index.htm;
proxy_pass http://registry;
#proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Authorization "";
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 8k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k; #如果系統很忙的時候可以申請更大的proxy_buffers 官方推薦*2
proxy_temp_file_write_size 64k; #proxy緩存臨時文件的大小
}
location /_ping {
auth_basic off;
proxy_pass http://registry;
}
location /v1/_ping {
auth_basic off;
proxy_pass http://registry;
}
}
}
其中紅色標明的地方一定要注意
如果不添加add_header Docker-Distribution-Api-Version registry/2.0 always;
無法用https訪問私有倉庫
如果不註解掉
#proxy_set_header Host $http_host;
這個的話，私有倉庫可以用https登錄，但是push的時候會報錯，
The push refers to a repository [dockertest.hc360.org/ubuntu_v1] (len: 1)
af88597ec24b: Pushing
dial tcp 192.168.33.75:80: connection refused
不註解的話它連接的是真實ip地址，無法代理
這個具體含義，大家查下nginx 反向代理 proxy_set_header
3.4驗證配置
[root@localhost ~]# /usr/local/nginx/sbin/nginx -t
如果沒有錯誤的話，啓動nginx
[root@localhost ~]# /usr/local/nginx/sbin/nginx
四 搭建docker registry V2
4.1 更改docker配置文件
[root@localhost ~]# systemctl stop docker
[root@localhost ~]# vim /etc/sysconfig/docker
OPTIONS='--selinux-enabled --insecure-registry dockertest.xxxx.com'
更改成這樣，添加--insecure-registry dockertest.xxxx.com 紅色的是私有倉庫的域名，更前面生成密鑰時候的域名對應
配置密鑰文件
Mkdir -p /etc/docker/certs.d/hc.docker.io
Cp /etc/pki/CA/cacert.pem ./hc.docker.io/ca-certificates.crt
cat /etc/pki/CA/cacert.pem >> /etc/pki/tls/certs/ca-bundle.crt
[root@localhost ~]#systemctl restart docker
[root@localhost ~]#vim config.yml
version: 0.1
log:
level: debug
formatter: text
fields:
service: registry
environment: staging
storage:
delete:
enabled: true
cache:
layerinfo: inmemory
filesystem:
rootdirectory: /var/lib/registry
http:
addr: :5000
secret: admin
[root@localhost ~]#mkdir data
docker run -d -p 5000:5000 --restart=always --name registry -v `pwd`/config.yml:/etc/docker/registry/config.yml -v `pwd`/data:/var/lib/registry registry:2
4.2 驗證
[root@localhost ~]# curl https://admin:[email protected]/v2/
結果返回{}
登錄
[root@localhost ~]docker login https://dockertest.xxxx.com
Username: admin
Password: #admin
Email: [email protected]
WARNING: login credentials saved in /root/.docker/config.json
Login Succeeded
4.3 驗證push鏡像到私有倉庫
在登錄的前題下，本機驗證
[root@localhost ~]# docker pull centos
[root@localhost ~]# docker images
daocloud.io/centos latest 60e65a8e4030 2 weeks ago 196.6 MB
[root@localhost ~]# docker tag 60e65a8e4030 dockertest.hc360.org/centos_test_v12
[root@localhost ~]# docker push dockertest.xxxx.com/centos_test_v12
The push refers to a repository [dockertest.xxxx.com/centos_test_v12] (len: 1)
60e65a8e4030: Pushed
60e65a8e4030: Preparing
838c1c5c4f83: Pushed
838c1c5c4f83: Preparing
latest: digest: sha256:3019f69254dcceb22a65859a3a414386ffbdd2908afba1b39e73220845c6cec8 size: 7228
查看push結果
[root@localhost ~]# curl https://admin:[email protected]/v2/_catalog
{"repositories":["centos_test_v12"]}
五 在client端測試
5.1 配置密鑰
[root@localhost ~]#scp /etc/pki/CA/cacert.pem [email protected]:/root/
在192.168.153.86這臺服務器上
[root@docker-minion2 ~]# cat /root/cacert.pem >> /etc/pki/tls/certs/ca-bundle.crt
[root@docker-minion2 ~]#vim /etc/sysconfig/docker
OPTIONS='--selinux-enabled --insecure-registry dockertest.xxxx.com'
[root@docker-minion2 ~]# systemctl restart docker
最好重啓下服務器吧，我第一次更改完無法登錄。重啓後可以登錄了，可能是沒有加載密鑰
[root@docker-minion2 ~]# docker login https://dockertest.hc360.org
Username: admin
Password:
Email: [email protected]
WARNING: login credentials saved in /root/.docker/config.json
Login Succeeded
5.2 驗證
和上面驗證方法一樣
能登錄上傳基本沒有什麼問題