swift对接整合ceph

基本原理科普：

       ceph对象存储组件radosgw原生支持swift接口，对接只是把openstack的权限认证配置到ceph里，创建endpoint时指向ceph rgw地址就可以了。

我们要做两件事儿

第一，安装ceph rgw。

        如果你是使用ceph deploy部署推荐看官方文档：

        https://docs.ceph.com/docs/mimic/install/install-ceph-gateway/

        如果你是手动部署的ceph，请往下看。

第二，安装python2-swiftclient，配置keystone认证，创建endpoint。

        官方文档：https://docs.ceph.com/docs/mimic/radosgw/keystone/

        懒得看英文的也请往下看。

 

笔者环境:openstack rocky aio，IP 192.168.0.1

ceph mimic 3节点，mon，osd均已正常启动，未安装rgw。IP ceph1:192.168.0.2,ceph2:192.168.0.3,ceph3:192.168.0.4

首先安装rgw：

安装rgw ，三个ceph节点上都执行

yum -y install ceph-radosgw

创建rgw需要的pool（根据环境调整pg和pgp数量），在ceph1上执行

pool=".rgw .rgw.root .rgw.control .rgw.gc .rgw.buckets .rgw.buckets.index .rgw.buckets.extra .log .intent-log .usage .users .users.email .users.swift .users.uid default.rgw.control default.rgw.data.root default.rgw.gc default.rgw.log"

PG_NUM=16

PGP_NUM=16

SIZE=3

for i in $pool

        do

        ceph osd pool create $i $PG_NUM

        ceph osd pool set $i size $SIZE

        done

for i in $pool

        do

        ceph osd pool set $i pgp_num $PGP_NUM

        done

在ceph1上创建keying，在ceph1上执行

cd /etc/ceph
sudo ceph-authtool --create-keyring /etc/ceph/ceph.client.radosgw.keyring
sudo chown ceph:ceph /etc/ceph/ceph.client.radosgw.keyring
sudo ceph-authtool /etc/ceph/ceph.client.radosgw.keyring -n client.rgw.ceph1 --gen-key
sudo ceph-authtool /etc/ceph/ceph.client.radosgw.keyring -n client.rgw.ceph2 --gen-key
sudo ceph-authtool /etc/ceph/ceph.client.radosgw.keyring -n client.rgw.ceph3 --gen-key
sudo ceph-authtool -n client.rgw.ceph1 --cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.client.radosgw.keyring
sudo ceph-authtool -n client.rgw.ceph2 --cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.client.radosgw.keyring
sudo ceph-authtool -n client.rgw.ceph3 --cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.client.radosgw.keyring
sudo ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.rgw.ceph1 -i /etc/ceph/ceph.client.radosgw.keyring
sudo ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.rgw.ceph2 -i /etc/ceph/ceph.client.radosgw.keyring
sudo ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.rgw.ceph3 -i /etc/ceph/ceph.client.radosgw.keyring

将创建好的keying拷贝到其它两台ceph节点，在ceph1上执行

scp /etc/ceph/ceph.client.radosgw.keyring root@ceph2:/etc/ceph/ceph.client.radosgw.keyring
scp /etc/ceph/ceph.client.radosgw.keyring root@ceph3:/etc/ceph/ceph.client.radosgw.keyring 

追加ceph的配置项,在所有ceph节点执行

echo "
[client.rgw.ceph1]
host=ceph1
keyring=/etc/ceph/ceph.client.radosgw.keyring
log file=/var/log/radosgw/client.radosgw.gateway.log
rgw_frontends = civetweb port=8081
rgw keystone api version = 3
rgw keystone url = 192.168.0.1:5000   #keystone认证服务的地址
rgw keystone accepted roles = admin
rgw keystone token cache size = 500
rgw keystone revocation interval = 300
rgw keystone implicit tenants = true
rgw s3 auth use keystone = true
rgw keystone admin user = admin
rgw keystone admin password = 123456 #你的admin用户的密码
rgw keystone admin tenant = admin
rgw keystone admin domain = Default
# 这里如果不设置，swift api无法识别url中的{account}，并且无法使用共享对象功能
rgw swift account in url = true


[client.rgw.ceph2]
host=ceph2
keyring=/etc/ceph/ceph.client.radosgw.keyring
log file=/var/log/radosgw/client.radosgw.gateway.log
rgw_frontends = civetweb port=8081
rgw keystone api version = 3
rgw keystone url = 192.168.0.1:5000   #keystone认证服务的地址
rgw keystone accepted roles = admin
rgw keystone token cache size = 500
rgw keystone revocation interval = 300
rgw keystone implicit tenants = true
rgw s3 auth use keystone = true
rgw keystone admin user = admin
rgw keystone admin password = 123456 #你的admin用户的密码
rgw keystone admin tenant = admin
rgw keystone admin domain = Default
# 这里如果不设置，swift api无法识别url中的{account}，并且无法使用共享对象功能
rgw swift account in url = true


[client.rgw.ceph3]
host=ceph3
keyring=/etc/ceph/ceph.client.radosgw.keyring
log file=/var/log/radosgw/client.radosgw.gateway.log
rgw_frontends = civetweb port=8081
rgw keystone api version = 3
rgw keystone url = 192.168.0.1:5000   #keystone认证服务的地址
rgw keystone accepted roles = admin
rgw keystone token cache size = 500
rgw keystone revocation interval = 300
rgw keystone implicit tenants = true
rgw s3 auth use keystone = true
rgw keystone admin user = admin
rgw keystone admin password = 123456 #你的admin用户的密码
rgw keystone admin tenant = admin
rgw keystone admin domain = Default
# 这里如果不设置，swift api无法识别url中的{account}，并且无法使用共享对象功能
rgw swift account in url = true

">> /etc/ceph/ceph.conf

启动radosgw服务，在所有ceph节点执行

hostname=`hostname`
systemctl start ceph-radosgw@rgw.$hostname
systemctl enable ceph-radosgw@rgw.$hostname

检查rgw服务是否成功启动

netstat -ntlp|grep radosgw
tcp        0      0 0.0.0.0:8081            0.0.0.0:*               LISTEN      42166/radosgw     

创建openstack的endpoint，在openstack机器上执行。

openstack service create --name swift --description "OpenStack Object Storage" object-store

#注意，我这写的是ceph1的地址，所有swift请求都会发往ceph1，读者可以通过haproxy等负载均衡工具进行配置，使三个节点变成高可用环境
openstack endpoint create --region RegionOne object-store public "http://192.168.0.2:8081/swift/v1" 

openstack endpoint create --region RegionOne object-store internal "http://192.168.0.2:8081/swift/v1" 

openstack endpoint create --region RegionOne object-store admin "http://192.168.0.2:8081/swift/v1" 

理论上来讲这里就算成功了，测试一下，在openstack机器上执行

#导入环境变量
export OS_USERNAME=admin
export OS_PASSWORD=123456
export OS_PROJECT_NAME=admin
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_AUTH_URL=http://192.168.0.1:5000/v3
export OS_IDENTITY_API_VERSION=3
#上传一个文件测试
swift upload testfile /var/log/messages
#查看上传的文件
swift list

最后，有遇到什么问题或者报错请留言，转载请注明出处，原创不易，谢谢配合。