轉載自:https://blog.csdn.net/huwh_/article/details/82286934
1. confd的部署
以下Linux系統爲例。
下載confd的二進制文件,下載地址爲:https://github.com/kelseyhightower/confd/releases。例如:
# Download the binary wget https://github.com/kelseyhightower/confd/releases/download/v0.16.0/confd-0.16.0-linux-amd64
# 重命名二進制文件,並移動到PATH的目錄下 mv confd-0.16.0-linux-amd64 /usr/local/bin/confd chmod +x /usr/local/bin/confd
# 驗證是否安裝成功 confd --help
2. confd的配置
Confd通過讀取後端存儲的配置信息來動態更新對應的配置文件,對應的後端存儲可以是etcd,redis等,其中etcd的v3版本對應的存儲後端爲etcdv3。
2.1. 創建confdir
confdir底下包含兩個目錄:
conf.d:confd的配置文件,主要包含配置的生成邏輯,例如模板源,後端存儲對應的keys,命令執行等。
templates:配置模板Template,即基於不同組件的配置,修改爲符合 Golang text templates的模板文件。
sudo mkdir -p /etc/confd/{conf.d,templates}
2.2. Template Resources
模板源配置文件是TOML格式的文件,主要包含配置的生成邏輯,例如模板源,後端存儲對應的keys,命令執行等。默認目錄在/etc/confd/conf.d。
參數說明:
必要參數
- dest (string) - The target file.
- keys (array of strings) - An array of keys.
- src (string) - The relative path of a configuration template.
可選參數
- gid (int) - The gid that should own the file. Defaults to the effective gid.
- mode (string) - The permission mode of the file.
- uid (int) - The uid that should own the file. Defaults to the effective uid.
- reload_cmd (string) - The command to reload config.
- check_cmd (string) - The command to check config. Use {{.src}} to reference the rendered source template.
- prefix (string) - The string to prefix to keys.
例子
例如:/etc/confd/conf.d/myapp-nginx.toml
[template]
prefix = "/myapp"
src = "nginx.tmpl"
dest = "/tmp/myapp.conf"
owner = "nginx"
mode = "0644"
keys = [
"/services/web"
]
check_cmd = "/usr/sbin/nginx -t -c {{.src}}"
reload_cmd = "/usr/sbin/service nginx reload"
2.3. Template
Template定義了單一應用配置的模板,默認存儲在/etc/confd/templates目錄下,模板文件符合Go的text/template格式。
模板文件常用函數有base,get,gets,lsdir,json等。具體可參考https://github.com/kelseyhightower/confd/blob/master/docs/templates.md。
例子:
/etc/confd/templates/nginx.tmpl
{{range $dir := lsdir "/services/web"}}
upstream {{base $dir}} {
{{$custdir := printf "/services/web/%s/*" $dir}}{{range gets $custdir}}
server {{$data := json .Value}}{{$data.IP}}:80;
{{end}}
}
server {
server_name {{base $dir}}.example.com;
location / {
proxy_pass {{base $dir}};
}
}
{{end}}
3. 創建後端存儲的配置數據
以etcdv3存儲爲例,在etcd中創建以下數據。
etcdctl --endpoints=$endpoints put /services/web/cust1/2 '{"IP": "10.0.0.2"}'
etcdctl --endpoints=$endpoints put /services/web/cust2/2 '{"IP": "10.0.0.4"}'
etcdctl --endpoints=$endpoints put /services/web/cust2/1 '{"IP": "10.0.0.3"}'
etcdctl --endpoints=$endpoints put /services/web/cust1/1 '{"IP": "10.0.0.1"}'
4. 啓動confd的服務
confd支持以daemon或者onetime兩種模式運行,當以daemon模式運行時,confd會監聽後端存儲的配置變化,並根據配置模板動態生成目標配置文件。
如果以daemon模式運行,則執行以下命令:
confd -watch -backend etcdv3 -node http://172.16.5.4:12379 &
# 執行命令
confd -onetime -backend etcdv3 -node http://172.16.5.4:12379
# output
2018-05-11T18:04:59+08:00 k8s-dbg-master-1 confd[35808]: INFO Backend set to etcdv3
2018-05-11T18:04:59+08:00 k8s-dbg-master-1 confd[35808]: INFO Starting confd
2018-05-11T18:04:59+08:00 k8s-dbg-master-1 confd[35808]: INFO Backend source(s) set to http://172.16.5.4:12379
2018-05-11T18:04:59+08:00 k8s-dbg-master-1 confd[35808]: INFO /root/myapp/twemproxy/conf/twemproxy.conf has md5sum 6f0f43abede612c75cb840a4840fbea3 should be 32f48664266e3fd6b56ee73a314ee272
2018-05-11T18:04:59+08:00 k8s-dbg-master-1 confd[35808]: INFO Target config /root/myapp/twemproxy/conf/twemproxy.conf out of sync
2018-05-11T18:04:59+08:00 k8s-dbg-master-1 confd[35808]: INFO Target config /root/myapp/twemproxy/conf/twemproxy.conf has been updated
5. 查看生成的配置文件
在/etc/confd/conf.d/myapp-nginx.toml中定義的配置文件的生成路徑爲/tmp/myapp.conf。
[root@k8s-dbg-master-1 dest]# cat myapp.conf
upstream cust1 {
server 10.0.0.1:80;
server 10.0.0.2:80;
}
server {
server_name cust1.example.com;
location / {
proxy_pass cust1;
}
}
upstream cust2 {
server 10.0.0.3:80;
server 10.0.0.4:80;
}
server {
server_name cust2.example.com;
location / {
proxy_pass cust2;
}
}
6. confd動態更新twemproxy
6.1. twemproxy.toml
confd的模板源文件配置:/etc/confd/conf.d/twemproxy.toml
[template]
src = "twemproxy.tmpl"
dest = "/root/myapp/twemproxy/conf/twemproxy.conf"
keys = [
"/twemproxy/pool"
]
check_cmd = "/usr/local/bin/nutcracker -t -c /root/myapp/twemproxy/conf/twemproxy.conf"
reload_cmd = "bash /root/myapp/twemproxy/reload.sh"
6.2. twemproxy.tmpl
模板文件:/etc/confd/templates/twemproxy.tmpl
global:
worker_processes: 4 # 併發進程數, 如果爲0, 這 fallback 回原來的單進程模型(不支持 config reload!)
user: nobody # worker 進程的用戶, 默認 nobody. 只要主進程是 root 用戶啓動才生效.
group: nobody # worker 進程的用戶組
worker_shutdown_timeout: 30 # 單位爲秒. 用於 reload 過程中在改時間段之後強制退出舊的 worker 進程.
pools: {{range gets "/twemproxy/pool/*"}}
{{base .Key}}: {{$pool := json .Value}}
listen: {{$pool.ListenAddr.IP}}:{{$pool.ListenAddr.Port}}
hash: fnv1a_64 # 選擇實例的 hash 規則
distribution: ketama
auto_eject_hosts: true # server 有問題是否剔除
redis: true # 是否爲 Redis 協議
{{if $pool.Password}}redis_auth: {{$pool.Password}}{{end}}
server_retry_timeout: 5000 # 被剔除多長時間後會重試
server_connections: 25 # NOTE: server 連接池的大小, 默認爲 1, 建議調整
server_failure_limit: 5 # 失敗多少次後暫時剔除
timeout: 1000 # Server 超時時間, 1 sec
backlog: 1024 # 連接隊列大小
preconnect: true # 預連接大小
servers:{{range $server := $pool.Servers}}
- {{$server.IP}}:{{$server.Port}}:1 {{if $server.Master}}master{{end}}
{{end}}
{{end}}
6.3. etcd中的配置格式
etcd中的配置通過一個map來定義爲完整的配置內容。其中key是twemproxy中pool的名稱,value是pool的所有內容。
配置對應go結構體如下:
type Pool struct{
ListenAddr ListenAddr `json:"ListenAddr,omitempty"`
Servers []Server `json:"Servers,omitempty"`
Password string `json:"Password,omitempty"`
}
type ListenAddr struct {
IP string `json:"IP,omitempty"`
Port string `json:"Port,omitempty"`
}
type Server struct {
IP string `json:"IP,omitempty"`
Port string `json:"Port,omitempty"`
Master bool `json:"Master,omitempty"`
}
配置對應JSON格式如下:
{
"ListenAddr": {
"IP": "192.168.5.7",
"Port": "22225"
},
"Servers": [
{
"IP": "10.233.116.168",
"Port": "6379",
"Master": true
},
{
"IP": "10.233.110.207",
"Port": "6379",
"Master": false
}
],
"Password": "987654"
}
6.4. 生成twemproxy配置文件
global:
worker_processes: 4 # 併發進程數, 如果爲0, 這 fallback 回原來的單進程模型(不支持 config reload!)
user: nobody # worker 進程的用戶, 默認 nobody. 只要主進程是 root 用戶啓動才生效.
group: nobody # worker 進程的用戶組
worker_shutdown_timeout: 30 # 單位爲秒. 用於 reload 過程中在改時間段之後強制退出舊的 worker 進程.
pools:
redis1:
listen: 192.168.5.7:22223
hash: fnv1a_64 # 選擇實例的 hash 規則
distribution: ketama
auto_eject_hosts: true # server 有問題是否剔除
redis: true # 是否爲 Redis 協議
redis_auth: 987654
server_retry_timeout: 5000 # 被剔除多長時間後會重試
server_connections: 25 # NOTE: server 連接池的大小, 默認爲 1, 建議調整
server_failure_limit: 5 # 失敗多少次後暫時剔除
timeout: 1000 # Server 超時時間, 1 sec
backlog: 1024 # 連接隊列大小
preconnect: true # 預連接大小
servers:
- 10.233.116.169:6379:1
redis2:
listen: 192.168.5.7:22224
hash: fnv1a_64 # 選擇實例的 hash 規則
distribution: ketama
auto_eject_hosts: true # server 有問題是否剔除
redis: true # 是否爲 Redis 協議
redis_auth: 987654
server_retry_timeout: 5000 # 被剔除多長時間後會重試
server_connections: 25 # NOTE: server 連接池的大小, 默認爲 1, 建議調整
server_failure_limit: 5 # 失敗多少次後暫時剔除
timeout: 1000 # Server 超時時間, 1 sec
backlog: 1024 # 連接隊列大小
preconnect: true # 預連接大小
servers:
- 10.233.110.223:6379:1 master
- 10.233.111.21:6379:1
參考文章:
https://github.com/kelseyhightower/confd/blob/master/docs/installation.md
https://github.com/kelseyhightower/confd/blob/master/docs/quick-start-guide.md
https://github.com/kelseyhightower/confd/blob/master/docs/template-resources.md
https://github.com/kelseyhightower/confd/blob/master/docs/templates.md