Ceph塊設備(RBD)常用命令

創建塊設備映像

?

1	rbd create --size {megabytes} {pool-name}/{image-name}

實例：

?

1	[ceph-deploy@ceph-admin ~]$ rbd create --size 1024 foo

羅列塊設備映像

?

1	rbd ls {poolname}

實例：

?

1 2	[ceph-deploy@ceph-admin ~]$ rbd ls # rbd list foo

檢索映像信息

?

1	rbd info {pool-name}/{image-name}

實例：

?

1 2 3 4 5 6 7 8

[ceph-deploy@ceph-admin ~]$ rbd info foo rbd image 'foo':     size 1024 MB in 256 objects     order 22 (4096 kB objects)     block_name_prefix: rbd_data.d3a4238e1f29     format: 2     features: layering, exclusive-lock, object-map, fast-diff, deep-flatten     flags:

調整塊設備映像大小

?

1 2	rbd resize --size 2048 foo (to increase) rbd resize --size 2048 foo --allow-shrink (to decrease)

實例：

?

1 2 3 4 5 6 7 8 9 10

[ceph-deploy@ceph-admin ~]$ rbd resize --size 2048 foo Resizing image: 100% complete...done. [ceph-deploy@ceph-admin ~]$ rbd info foo rbd image 'foo':     size 2048 MB in 512 objects     order 22 (4096 kB objects)     block_name_prefix: rbd_data.d3a4238e1f29     format: 2     features: layering, exclusive-lock, object-map, fast-diff, deep-flatten     flags:

刪除塊設備映像

?

1	rbd rm {pool-name}/{image-name}

實例：

?

1 2	[ceph-deploy@ceph-admin ~]$ rbd rm foo Removing image: 100% complete...done.

映射塊設備

?

1	sudo rbd map {pool-name}/{image-name} --id {user-name}

實例：

?

1 2	[ceph-deploy@ceph-admin ~]$ sudo rbd map rbd/foo /dev/rbd0

查看已映射塊設備

?

1	rbd showmapped

實例：

?

1 2 3

[ceph-deploy@ceph-admin ~]$ rbd showmapped id pool image snap device    0  rbd  foo   -    /dev/rbd0

取消塊設備映射

?

1	sudo rbd unmap /dev/rbd/{poolname}/{imagename}

實例：

?

1	[ceph-deploy@ceph-admin ~]$ sudo rbd unmap /dev/rbd0

使用Ceph塊設備

創建並掛載一個文件系統：

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

[ceph-deploy@ceph-admin ~]$ sudo mkfs.xfs /dev/rbd0 meta-data=/dev/rbd0              isize=512    agcount=9, agsize=64512 blks          =                       sectsz=512   attr=2, projid32bit=1          =                       crc=1        finobt=0, sparse=0 data     =                       bsize=4096   blocks=524288, imaxpct=25          =                       sunit=1024   swidth=1024 blks naming   =version 2              bsize=4096   ascii-ci=0 ftype=1 log      =internal log           bsize=4096   blocks=2560, version=2          =                       sectsz=512   sunit=8 blks, lazy-count=1 realtime =none                   extsz=4096   blocks=0, rtextents=0 [ceph-deploy@ceph-admin ~]$ sudo mkdir /mnt/ceph-disk0 [ceph-deploy@ceph-admin ~]$ sudo mount /dev/rbd0 /mnt/ceph-disk0 [ceph-deploy@ceph-admin ~]$ df -h /mnt/ceph-disk0 Filesystem                 Size  Used Avail Use% Mounted on /dev/rbd0                  2.0G   33M  2.0G   2% /mnt/ceph-disk0

通過將數據寫入塊設備來進行檢測：

?

1 2 3 4 5 6 7 8 9

[ceph-deploy@ceph-admin ~]$ sudo dd if=/dev/zero of=/mnt/ceph-disk0/file0 count=100 bs=1M 100+0 records in 100+0 records out 104857600 bytes (105 MB) copied, 0.779733 s, 134 MB/s [ceph-deploy@ceph-admin ~]$ sudo ls /mnt/ceph-disk0/file0 -lh -rw-r--r--. 1 root root 100M Jul 19 15:33 /mnt/ceph-disk0/file0 [ceph-deploy@ceph-admin ~]$ df -h /mnt/ceph-disk0 Filesystem                 Size  Used Avail Use% Mounted on /dev/rbd0                  2.0G  133M  1.9G   7% /mnt/ceph-disk0

增加塊設備映像大小後，擴展文件系統來利用增加了的存儲空間

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

[ceph-deploy@ceph-admin ~]$ sudo dmesg | grep -i capacity [17945.781304] rbd: rbd0: capacity 2147483648 features 0x1 [21955.926051] rbd0: detected capacity change from 2147483648 to 4294967296 [ceph-deploy@ceph-admin ~]$ sudo xfs_growfs -d /mnt/ceph-disk0/ meta-data=/dev/rbd0              isize=512    agcount=9, agsize=64512 blks          =                       sectsz=512   attr=2, projid32bit=1          =                       crc=1        finobt=0 spinodes=0 data     =                       bsize=4096   blocks=524288, imaxpct=25          =                       sunit=1024   swidth=1024 blks naming   =version 2              bsize=4096   ascii-ci=0 ftype=1 log      =internal               bsize=4096   blocks=2560, version=2          =                       sectsz=512   sunit=8 blks, lazy-count=1 realtime =none                   extsz=4096   blocks=0, rtextents=0 data blocks changed from 524288 to 1048576 [ceph-deploy@ceph-admin ~]$ df -h /mnt/ceph-disk0 Filesystem                 Size  Used Avail Use% Mounted on /dev/rbd0                  4.0G  133M  3.9G   4% /mnt/ceph-disk0

可能遇到的錯誤：

?

1 2 3 4 5

[ceph-deploy@ceph-admin ~]$ sudo rbd map rbd/foo rbd: sysfs write failed RBD image feature set mismatch. You can disable features unsupported by the kernel with "rbd feature disable". In some cases useful info is found in syslog - try "dmesg | tail" or so. rbd: map failed: (6) No such device or address

原因：

rbd鏡像的一些特性，OS kernel並不支持，所以映射失敗。

查看該鏡像支持了哪些特性：

?

1 2 3 4 5 6 7 8

[ceph-deploy@ceph-admin ~]$ rbd info foo rbd image 'foo':     size 2048 MB in 512 objects     order 22 (4096 kB objects)     block_name_prefix: rbd_data.d3a4238e1f29     format: 2     features: layering, exclusive-lock, object-map, fast-diff, deep-flatten     flags:

可以看到特性feature一欄，由於OS的kernel只支持layering，其他都不支持，所以需要把部分不支持的特性disable掉。

解決方案一：

直接diable這個rbd鏡像的不支持的特性：

?

1	[ceph-deploy@ceph-admin ~]$ rbd feature disable foo exclusive-lock object-map fast-diff deep-flatten

解決方案二：

創建rbd鏡像時就指明需要的特性，如：

?

1	[ceph-deploy@ceph-admin ~]$ rbd create --size 2048 foo --image-feature layering

解決方案三：

如果還想一勞永逸，那麼就在執行創建rbd鏡像命令的服務器中，修改Ceph配置文件/etc/ceph/ceph.conf，在global section下，增加
rbd_default_features = 1，再創建rdb鏡像：

?

1	<code><code><code>[ceph-deploy@ceph-admin ~]$ rbd create --size 2048 foo</code></code></code>

通過上述三種方法後，查看rbd鏡像的信息

?

1 2 3 4 5 6 7 8

<code><code><code>[ceph-deploy@ceph-admin ~]$ rbd info foo rbd image 'foo':     size 2048 MB in 512 objects     order 22 (4096 kB objects)     block_name_prefix: rbd_data.d3a4238e1f29     format: 2     features: layering     flags: </code></code></code>

再次嘗試映射rdb鏡像到本地塊設備，成功！

?

1 2	<code><code><code>[ceph-deploy@ceph-admin ~]$ sudo rbd map rbd/foo /dev/rbd0</code></code></code>