swift對接整合ceph

基本原理科普:

       ceph對象存儲組件radosgw原生支持swift接口,對接只是把openstack的權限認證配置到ceph裏,創建endpoint時指向ceph rgw地址就可以了。

我們要做兩件事兒

第一,安裝ceph rgw。

        如果你是使用ceph deploy部署推薦看官方文檔:

        https://docs.ceph.com/docs/mimic/install/install-ceph-gateway/

        如果你是手動部署的ceph,請往下看。

第二,安裝python2-swiftclient,配置keystone認證,創建endpoint。

        官方文檔:https://docs.ceph.com/docs/mimic/radosgw/keystone/

        懶得看英文的也請往下看。

 

筆者環境:openstack rocky aio,IP 192.168.0.1

ceph mimic 3節點,mon,osd均已正常啓動,未安裝rgw。IP ceph1:192.168.0.2,ceph2:192.168.0.3,ceph3:192.168.0.4

首先安裝rgw:

安裝rgw ,三個ceph節點上都執行

yum -y install ceph-radosgw

創建rgw需要的pool(根據環境調整pg和pgp數量),在ceph1上執行

pool=".rgw .rgw.root .rgw.control .rgw.gc .rgw.buckets .rgw.buckets.index .rgw.buckets.extra .log .intent-log .usage .users .users.email .users.swift .users.uid default.rgw.control default.rgw.data.root default.rgw.gc default.rgw.log"

PG_NUM=16

PGP_NUM=16

SIZE=3

for i in $pool

        do

        ceph osd pool create $i $PG_NUM

        ceph osd pool set $i size $SIZE

        done

for i in $pool

        do

        ceph osd pool set $i pgp_num $PGP_NUM

        done

在ceph1上創建keying,在ceph1上執行

cd /etc/ceph
sudo ceph-authtool --create-keyring /etc/ceph/ceph.client.radosgw.keyring
sudo chown ceph:ceph /etc/ceph/ceph.client.radosgw.keyring
sudo ceph-authtool /etc/ceph/ceph.client.radosgw.keyring -n client.rgw.ceph1 --gen-key
sudo ceph-authtool /etc/ceph/ceph.client.radosgw.keyring -n client.rgw.ceph2 --gen-key
sudo ceph-authtool /etc/ceph/ceph.client.radosgw.keyring -n client.rgw.ceph3 --gen-key
sudo ceph-authtool -n client.rgw.ceph1 --cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.client.radosgw.keyring
sudo ceph-authtool -n client.rgw.ceph2 --cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.client.radosgw.keyring
sudo ceph-authtool -n client.rgw.ceph3 --cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.client.radosgw.keyring
sudo ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.rgw.ceph1 -i /etc/ceph/ceph.client.radosgw.keyring
sudo ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.rgw.ceph2 -i /etc/ceph/ceph.client.radosgw.keyring
sudo ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.rgw.ceph3 -i /etc/ceph/ceph.client.radosgw.keyring

將創建好的keying拷貝到其它兩臺ceph節點,在ceph1上執行

scp /etc/ceph/ceph.client.radosgw.keyring root@ceph2:/etc/ceph/ceph.client.radosgw.keyring
scp /etc/ceph/ceph.client.radosgw.keyring root@ceph3:/etc/ceph/ceph.client.radosgw.keyring 

追加ceph的配置項,在所有ceph節點執行

echo "
[client.rgw.ceph1]
host=ceph1
keyring=/etc/ceph/ceph.client.radosgw.keyring
log file=/var/log/radosgw/client.radosgw.gateway.log
rgw_frontends = civetweb port=8081
rgw keystone api version = 3
rgw keystone url = 192.168.0.1:5000   #keystone認證服務的地址
rgw keystone accepted roles = admin
rgw keystone token cache size = 500
rgw keystone revocation interval = 300
rgw keystone implicit tenants = true
rgw s3 auth use keystone = true
rgw keystone admin user = admin
rgw keystone admin password = 123456 #你的admin用戶的密碼
rgw keystone admin tenant = admin
rgw keystone admin domain = Default
# 這裏如果不設置,swift api無法識別url中的{account},並且無法使用共享對象功能
rgw swift account in url = true


[client.rgw.ceph2]
host=ceph2
keyring=/etc/ceph/ceph.client.radosgw.keyring
log file=/var/log/radosgw/client.radosgw.gateway.log
rgw_frontends = civetweb port=8081
rgw keystone api version = 3
rgw keystone url = 192.168.0.1:5000   #keystone認證服務的地址
rgw keystone accepted roles = admin
rgw keystone token cache size = 500
rgw keystone revocation interval = 300
rgw keystone implicit tenants = true
rgw s3 auth use keystone = true
rgw keystone admin user = admin
rgw keystone admin password = 123456 #你的admin用戶的密碼
rgw keystone admin tenant = admin
rgw keystone admin domain = Default
# 這裏如果不設置,swift api無法識別url中的{account},並且無法使用共享對象功能
rgw swift account in url = true


[client.rgw.ceph3]
host=ceph3
keyring=/etc/ceph/ceph.client.radosgw.keyring
log file=/var/log/radosgw/client.radosgw.gateway.log
rgw_frontends = civetweb port=8081
rgw keystone api version = 3
rgw keystone url = 192.168.0.1:5000   #keystone認證服務的地址
rgw keystone accepted roles = admin
rgw keystone token cache size = 500
rgw keystone revocation interval = 300
rgw keystone implicit tenants = true
rgw s3 auth use keystone = true
rgw keystone admin user = admin
rgw keystone admin password = 123456 #你的admin用戶的密碼
rgw keystone admin tenant = admin
rgw keystone admin domain = Default
# 這裏如果不設置,swift api無法識別url中的{account},並且無法使用共享對象功能
rgw swift account in url = true

">> /etc/ceph/ceph.conf

啓動radosgw服務,在所有ceph節點執行

hostname=`hostname`
systemctl start ceph-radosgw@rgw.$hostname
systemctl enable ceph-radosgw@rgw.$hostname

檢查rgw服務是否成功啓動

netstat -ntlp|grep radosgw
tcp        0      0 0.0.0.0:8081            0.0.0.0:*               LISTEN      42166/radosgw     

創建openstack的endpoint,在openstack機器上執行。

openstack service create --name swift --description "OpenStack Object Storage" object-store

#注意,我這寫的是ceph1的地址,所有swift請求都會發往ceph1,讀者可以通過haproxy等負載均衡工具進行配置,使三個節點變成高可用環境
openstack endpoint create --region RegionOne object-store public "http://192.168.0.2:8081/swift/v1" 

openstack endpoint create --region RegionOne object-store internal "http://192.168.0.2:8081/swift/v1" 

openstack endpoint create --region RegionOne object-store admin "http://192.168.0.2:8081/swift/v1" 

理論上來講這裏就算成功了,測試一下,在openstack機器上執行

#導入環境變量
export OS_USERNAME=admin
export OS_PASSWORD=123456
export OS_PROJECT_NAME=admin
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_DOMAIN_NAME=Default
export OS_AUTH_URL=http://192.168.0.1:5000/v3
export OS_IDENTITY_API_VERSION=3
#上傳一個文件測試
swift upload testfile /var/log/messages
#查看上傳的文件
swift list

最後,有遇到什麼問題或者報錯請留言,轉載請註明出處,原創不易,謝謝配合。

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章