saltstack應用之批量部署mysql
應用環境：
使用salt寫了個自動部署mysql的配置配置管理文件，由於mysql的有版本號，端口等不確定屬性，需要使用pillar來單獨配置每個minion的屬性。
一，原始方法
例如，需要在salt id爲10.1.1.1-centos.game.web的服務器上部署mysql，版本號爲5.5.25，實例有3個，3306，3307,3308， 正常情況下的流程是這樣的：
1，在/srv/salt/top.sls中添加配置信息,確保mysql對應的配置被加載
base:
10.1.1.1-centos.game.web:
- centos.public_services.mysql
2,在/srv/pillar/top.sls配置minion對應的sls文件位置(ps:salt文件中不能再帶”.”，否則會報錯)
base:
10.1.1.1-centos.game.web:
- custom.10-1-1-1-centos-game-web
3,新建/srv/pillar/custom/10-1-1-1-centos-game-web.sls,內容如下:
mysql:
ports:
- 3306
- 3307
- 3308
version: '5_5_25'
4,執行同步命令
salt 10.1.1.1-centos.game.web state.highstate -v -t 300
二，改進方法
oh,shit,說好的自動化呢，怎麼還要這麼多步驟，這可不行！
利用py模式的sls配置文件（其實就是python腳本，只要返回yaml格式的字典文件就好了），我們可以將以上的操作簡化成1步，思路如下：
1，/srv/pillar/top.sls中編寫配置：
base:
'*':
- custom
2，使用py模式編寫/srv/pillar/custom/init.sls，自動讀取pillar配置，例如salt id是：10.1.1.1-centos.game.web，那麼project爲game，然後根據獲取的pillar_root組合成路徑/srv/pillar/custom/game/10.1.1.1-centos.game.web.yaml,利用yaml模塊從文件中讀取信息，返回字典
3，在/srv/salt/top.sls文件中匹配所有的minion
‘*’：
- centos.public_services
4，/srv/salt/centos/public_services/init.sls文件使用py模式編寫，配置會獲取對應的minion的pillar信息，如果包含mysql配置信息且配置正確的話，則返回mysql實例的配置。
那現在要怎麼使用呢，很簡單，例如你的id爲10.1.1.1-centos.game.web，首先在/srv/pillar/custom/目錄下建個game目錄(從salt id獲取的項目名)，然後在game目錄先新建文件10.1.1.1-centos.game.web.yaml,裏面寫上配置信息：
mysql:
ports:
- 3306
- 3307
- 3308
version: '5_5_25'
最後執行命令:
salt 10.1.1.1-centos.game.web state.highstate -v -t 300
靜靜的等待執行完成就好了！
三，具體代碼
/srv/pillar/custom/init.sls
#!py
#coding:utf-8
"""
返回minion對應的pillar信息
"""
import yaml
import os
def run():
"""
首先獲取請求的id，從id中獲取project，例如id是：1.2.3.4-centos.game.web，那麼project爲game
然後根據獲取的pillar_root組合成路徑/srv/pillar/custom/game/1.2.3.4-centos.game.web.yaml,利用yaml模塊從文件中讀取信息，返回字典
"""
config={}
id=__opts__['id']
project=id.split('-')[-1].split('.')[1]
pillar_root=__opts__['pillar_roots']['base'][0]
path='%s/custom/%s/%s.yaml'%(pillar_root,project,id)
if os.path.isfile(path):
s=open(path).read()
config=yaml.load(s)
return config
/srv/salt/centos/public_services/init.sls
#!py
#coding: utf-8
import subprocess
class MY_ERROR(Exception):
def __init__(self,value):
self.value=value
def __str__(self):
return self.value
def mysql():
"""
1,檢查是minion中pillar是否有mysql參數,以及參數是否合法
如果參數沒有問題，則返回對應版本的include配置
pillar e.q.
mysql:
ports:
- 3306
- 3307
version: '5_5_25'
"""
mysql_sls_path='centos.public_services.mysql.'
#必要的參數
required_keys=['version','ports']
if __pillar__.has_key('mysql'):
mysql_d=__pillar__['mysql']
#不存在必要的鍵值對則返回None
for key in required_keys:
if not mysql_d.has_key(key) or str(mysql_d[key]).strip()=="":
raise MY_ERROR('key error! key: %s'%(str(key)))
#判斷port是否合法
for port in mysql_d['ports']:
if not port or not 1024<int(port)<65535:
raise MY_ERROR('mysql ports value error: %s'%(str(mysql_d['ports'])))
#組合配置參數
cfg=mysql_sls_path+str(mysql_d['version'][0])
return cfg
return None
def run():
config={}
config['include']=[]
#mysql
mysql_cfg=mysql()
if mysql_cfg:
config['include'].append(mysql_cfg)
if config['include']==[]:
return {}
return config
/srv/salt/centos/public_services/mysql/5/init.sls
include:
- centos.public_services.mysql.5.packet
- centos.public_services.mysql.5.my_cnf
- centos.public_services.mysql.5.instance
/srv/salt/centos/public_services/mysql/5/packet.sls
mysql軟件包要放到相應的目錄中，具體位置參考下面配置
{%set version=pillar['mysql']['version']%}
mysql:
user.present:
- home: /home/mysql
- shell: /sbin/nologin
/usr/local/nagios/libexec/check_safe -u:
cmd.wait:
- watch:
- user: mysql
/usr/local/src/mysql-{{version.replace('_','.')}}.tar.gz:
file.managed:
- source: salt://centos/public_services/mysql/5/`version`/mysql-{{version.replace('_','.')}}.tar.gz
tar -xf mysql-{{version.replace('_','.')}}.tar.gz -C /usr/local/:
cmd.run:
- cwd: /usr/local/src
- unless: ls -l /usr/local/ | grep -e ".* mysql-{{version.replace('_','.')}}$"
/usr/local/mysql-{{version.replace('_','.')}}:
file.directory:
- user: mysql
- group: mysql
- recurse:
- user
- group
/data/mysql_log:
file.directory:
- makedirs: True
- user: mysql
- group: mysql
- recurse:
- user
- group
/data/log-bin:
file.directory:
- makedirs: True
- user: mysql
- group: mysql
- recurse:
- user
- group
/usr/bin/mysql:
file.symlink:
- target: /usr/local/mysql-{{version.replace('_','.')}}/bin/mysql
- unless: ls -l /usr/bin | grep -e " mysql$"
/srv/salt/centos/public_services/mysql/5/my_cnf.sls
注意修改最後的初始化密碼
my.cnf配置文件需要放到對應的目錄中，my.cnf文件中要設置對應的模板變量:
port= `port` #還有其它的和端口相關的配置都要改成`port`
basedir=/usr/local/mysql-`version` #還有其它的和版本相關的配置都要改成`version`
#!py
#coding:utf-8
"""
生成my.cnf配置文件，如果文件已存在，不作任何修改
"""
import os
def run():
config={}
version=__pillar__['mysql']['version']
for port in __pillar__['mysql']['ports']:
port=str(port)
if not os.path.isfile('/data/mysql_data_%s/my.cnf'%(port)):
config['/data/mysql_data_%s/my.cnf'%(port)]={
'file.managed':[
{'source':'salt://centos/public_services/mysql/5/%s/my.cnf'%(version)},
{'template':'jinja'},
{'context':{'port':port,'version':'%s'%(version.replace('_','.'))}},
{'require':[{'file':'/data/mysql_data_%s'%(port)}]},
],
}
config['chown mysql.mysql /data/mysql_data_%s/my.cnf'%(port)]='cmd.run'
return config
/srv/salt/centos/public_services/mysql/5/instance.sls
mysql啓動腳本需要放到對應的目錄中，啓動腳本中要設置對應的模板變量:
basedir=/usr/local/mysql-`version` #還有其它的和版本相關的配置都要改成`version`
datadir=/data/mysql_data_`port` #還有其它的和端口相關的配置都要改成`port`
{% for port in pillar['mysql']['ports'] %}
/data/mysql_data_`port`:
file.directory:
- makedirs: True
- user: mysql
- group: mysql
- recurse:
- user
- group
/etc/init.d/mysqld_`port`:
file.managed:
- source: salt://centos/public_services/mysql/5/{{pillar['mysql']['version']}}/mysql.service
- user: root
- group: root
- mode: 755
- template: jinja
- context:
port: `port`
version: {{pillar['mysql']['version'].replace('_','.')}}
chkconfig --add mysqld_`port`;chkconfig --level 345 mysqld_`port` on;:
cmd.run:
- unless: chkconfig --list | grep mysqld_`port`
#初始化庫
init_mysql_`port`:
cmd.run:
- name: /usr/local/mysql-{{pillar['mysql']['version'].replace('_','.')}}/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql-{{pillar['mysql']['version'].replace('_','.')}} --datadir=/data/mysql_data_`port`/
- unless: ls -l /data/mysql_data_`port` | grep -e ".* mysql$"
/etc/init.d/mysqld_`port` start:
cmd.wait:
- watch:
- cmd: init_mysql_`port`
/usr/local/mysql-{{pillar['mysql']['version'].replace('_','.')}}/bin/mysqladmin -uroot -S /tmp/mysql_`port`.sock password 'yourmysqlpasswd!':
cmd.wait:
- watch:
- cmd: init_mysql_`port`
{%endfor%}
四，總結
這個方法不僅可以用在mysql，同樣的nginx，redis等都可以舉一反三，配置文件編寫完成後，僅僅需要簡單的在pillar中添加幾