在ceph监视器上执行
CINDER_PASSWD='cinder1234!'
controllerHost='controller'
RABBIT_PASSWD='0penstackRMQ'
1.创建pool池
为cinder-volume服务创建pool池(因为我只有一个OSD节点，所以要将副本数设置为1)
ceph osd pool create cinder-volumes 32
ceph osd pool set cinder-volumes size 1
ceph osd pool application enable cinder-volumes rbd
ceph osd lspools
2.查看pool池的使用情况
ceph df
3.创建账号
ceph auth get-or-create client.cinder-volumes mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=cinder-volumes, allow rwx pool=glance-images' -o /etc/ceph/ceph.client.cinder-volumes.keyring
#查看
ceph auth ls | grep -EA3 'client.(cinder-volumes)'
4.修改ceph.conf配置文件并同步到所有的监视器节点（这步一定要操作）
su - cephd
cd ~/ceph-cluster/
cat <<EOF>> ceph.conf
[client.cinder-volumes]
keyring = /etc/ceph/ceph.client.cinder-volumes.keyring
EOF
ceph-deploy --overwrite-conf admin ceph-mon01
exit
5.安装cinder-volume组件和ceph客户端(如果ceph监视器是在控制节点上不需要执行这一步)
yum -y install openstack-cinder python-keystone ceph-common
6.使用uuidgen生成一个uuid(确保cinder和libvirt中的UUID一致)
uuidgen
运行uuidgen命令可以得到下面的UUID值：086037e4-ad59-4c61-82c9-86edc31b0bc0
7.配置cinder-volume服务与cinder-api服务进行交互
openstack-config --set /etc/cinder/cinder.conf DEFAULT transport_url rabbit://openstack:${RABBIT_PASSWD}@${controllerHost}:5672
openstack-config --set /etc/cinder/cinder.conf cache backend oslo_cache.memcache_pool
openstack-config --set /etc/cinder/cinder.conf cache enabled true
openstack-config --set /etc/cinder/cinder.conf cache memcache_servers ${controllerHost}:11211
openstack-config --set /etc/cinder/cinder.conf DEFAULT auth_strategy keystone
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken auth_uri http://${controllerHost}:5000
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken auth_url http://${controllerHost}:5000
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken auth_type password
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken project_domain_id default
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken user_domain_id default
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken project_name service
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken username cinder
openstack-config --set /etc/cinder/cinder.conf keystone_authtoken password ${CINDER_PASSWD}
openstack-config --set /etc/cinder/cinder.conf oslo_concurrency lock_path /var/lib/cinder/tmp
8.配置cinder-volume服务使用的后端存储为ceph
openstack-config --set /etc/cinder/cinder.conf DEFAULT enabled_backends ceph
9.配置cinder-volume服务驱动ceph
openstack-config --set /etc/cinder/cinder.conf ceph volume_driver cinder.volume.drivers.rbd.RBDDriver
openstack-config --set /etc/cinder/cinder.conf ceph rbd_pool cinder-volumes
openstack-config --set /etc/cinder/cinder.conf ceph rbd_user cinder-volumes
openstack-config --set /etc/cinder/cinder.conf ceph rbd_ceph_conf /etc/ceph/ceph.conf
openstack-config --set /etc/cinder/cinder.conf ceph rbd_flatten_volume_from_snapshot false
openstack-config --set /etc/cinder/cinder.conf ceph bd_max_clone_depth 5
openstack-config --set /etc/cinder/cinder.conf ceph rbd_store_chunk_size 4
openstack-config --set /etc/cinder/cinder.conf ceph rados_connect_timeout -1
openstack-config --set /etc/cinder/cinder.conf ceph glance_api_version 2
openstack-config --set /etc/cinder/cinder.conf ceph rbd_secret_uuid 086037e4-ad59-4c61-82c9-86edc31b0bc0
10.启动cinder-volume服务
systemctl enable openstack-cinder-volume.service
systemctl start openstack-cinder-volume.service
systemctl status openstack-cinder-volume.service
在需要挂载ceph卷的所有计算节点上执行
1.创建secret文件(UUID需要与cinder服务中一致)
cat << EOF > ~/secret.xml
<secret ephemeral='no' private='no'>
<uuid>086037e4-ad59-4c61-82c9-86edc31b0bc0</uuid>
<usage type='ceph'>
<name>client.cinder-volumes secret</name>
</usage>
</secret>
EOF
2.从ceph监视器上获取cinder-volumes账户的密钥环
ceph auth get-key client.cinder-volumes
得到如下的结果：
AQCxfDFdgp2qKRAAUY/vep29N39Qv7xWKYqMUw==
3.在libvirt中注册UUID
virsh secret-define --file ~/secret.xml
4.在libvirt中添加UUID和cinder-volumes密钥环
virsh secret-set-value --secret 086037e4-ad59-4c61-82c9-86edc31b0bc0 --base64 AQCxfDFdgp2qKRAAUY/vep29N39Qv7xWKYqMUw==
5.查看libvirt中添加的UUID
virsh secret-list
6.重启libvirt
systemctl restart libvirtd.service
systemctl status libvirtd.service
出错回滚的方案
1.删除pool池
先在所有的监视器节点上开启删除pool的权限，然后才可以删除。
删除pool时ceph要求必须输入两次pool名称，同时加上--yes-i-really-really-mean-it选项。
echo '
mon_allow_pool_delete = true
[mon]
mon allow pool delete = true
' >> /etc/ceph/ceph.conf
systemctl restart ceph-mon.target
ceph osd pool delete cinder-volumes cinder-volumes --yes-i-really-really-mean-it
2.删除账号
ceph auth del client.cinder-volumes
3.删除libvirt中注册的UUID和cinder-volumes密钥环
查看：
virsh secret-list
删除(secret-undefine后跟uuid值)：
virsh secret-undefine 086037e4-ad59-4c61-82c9-86edc31b0bc0