對於一個存儲,不可避免的會遇到擴容、縮減存儲容量的問題。
1 爲Glusterfs擴容
提前準備好了一個gluster volume:
[root@node01 ~]# gluster volume info repvol Volume Name: repvol Type: Replicate Volume ID: 7015b7fe-8039-4d5d-b698-e8321df5289c Status: Started Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: node01.lab.example.com:/bricks/thinvol1/brick Brick2: node02.lab.example.com:/bricks/thinvol1/brick Options Reconfigured: performance.stat-prefetch: off storage.batch-fsync-delay-usec: 0 server.allow-insecure: on performance.readdir-ahead: on transport.address-family: inet nfs.disable: on
可以看出repvol是一個包含2個brick的1x2複製卷,將repvol掛載到node02 server上,並寫入一些數據以作測試。
[root@node02 ~]# mount -t cifs //node01.lab.example.com/gluster-repvol -o user=smbuser,pass=redhat /mnt
[root@node02 ~]# cd /mnt [root@node02 mnt]# ls [root@node02 mnt]# touch {1..100}.file
使用如下命令爲repvol再增加兩個brick
[root@node01 ~]# gluster volume add-brick repvol node01.lab.example.com:/bricks/thinvol2/brick node02.lab.example.com:/bricks/thinvol2/brick volume add-brick: success
[root@node01 ~]# gluster volume info repvol Volume Name: repvol Type: Distributed-Replicate Volume ID: 7015b7fe-8039-4d5d-b698-e8321df5289c Status: Started Snapshot Count: 0 Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: node01.lab.example.com:/bricks/thinvol1/brick Brick2: node02.lab.example.com:/bricks/thinvol1/brick Brick3: node01.lab.example.com:/bricks/thinvol2/brick Brick4: node02.lab.example.com:/bricks/thinvol2/brick Options Reconfigured: performance.stat-prefetch: off storage.batch-fsync-delay-usec: 0 server.allow-insecure: on performance.readdir-ahead: on transport.address-family: inet nfs.disable: on
可以看到repvol已經變成了一個2x2的分佈式複製卷,擴容後我們還需要把原有的數據rebalance
[root@node01 ~]# gluster volume rebalance repvol start volume rebalance: repvol: success: Rebalance on repvol has been started successfully. Use rebalance status command to check status of the rebalance process. ID: a8fba779-6c2d-449b-8847-58aeb2ae1798
[root@node01 ~]# gluster volume rebalance repvol status Node Rebalanced-files size scanned failures skipped status run time in h:m:s --------- ----------- ----------- ----------- ----------- ----------- ------------ -------------- localhost 48 0Bytes 100 0 0 completed 0:0:5 node02 0 0Bytes 0 0 0 completed 0:0:2 volume rebalance: repvol: success
這裏有一個需要注意的地方,當數據量太大的時候,對數據進行rebalance必須要考慮的一個問題就是性能,不能因爲數據rebalance而影響我們的存儲的正常使用。Glusterfs也考慮到了這個問題,在進行數據rebalance時,根據實際場景不同設計了三種不同的“級別”:
1)lazy:每次僅可以遷移一個文件
2)normal:默認設置,每次遷移2個文件或者是(CPU邏輯個數-4)/2,哪個大,選哪個
3)aggressive:每次遷移4個文件或者是(CPU邏輯個數-4)/2
通過以下命令進行配置:
gluster volume set VOLUME-NAME cluster.rebal-throttle [lazy|normal|aggressive]
如將volume repvol設置爲lazy
[root@node01 ~]# gluster volume set repvol cluster.rebal-throttle lazy volume set: success
2 縮減volume大小
將我們剛剛添加的2個brick再從volume repovl中刪掉
[root@node01 ~]# gluster volume remove-brick repvol node01.lab.example.com:/bricks/thinvol2/brick node02.lab.example.com:/bricks/thinvol2/brick start volume remove-brick start: success ID: db72aaf8-e60d-4e56-be60-54469df9c233
[root@node01 ~]# gluster volume remove-brick repvol node01.lab.example.com:/bricks/thinvol2/brick node02.lab.example.com:/bricks/thinvol2/brick status Node Rebalanced-files size scanned failures skipped status run time in h:m:s --------- ----------- ----------- ----------- ----------- ----------- ------------ -------------- localhost 48 0Bytes 101 0 0 completed 0:0:3 node02 0 0Bytes 0 0 0 completed 0:0:1
[root@node01 ~]# gluster volume remove-brick repvol node01.lab.example.com:/bricks/thinvol2/brick node02.lab.example.com:/bricks/thinvol2/brick commit Removing brick(s) can result in data loss. Do you want to Continue? (y/n) y volume remove-brick commit: success Check the removed bricks to ensure all files are migrated. If files with data are found on the brick path, copy them via a gluster mount point before re-purposing the removed brick.
此時再看repvol已經重新變爲1x2的複製捲了
[root@node01 ~]# gluster volume info repvol Volume Name: repvol Type: Replicate Volume ID: 7015b7fe-8039-4d5d-b698-e8321df5289c Status: Started Snapshot Count: 0 Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: node01.lab.example.com:/bricks/thinvol1/brick Brick2: node02.lab.example.com:/bricks/thinvol1/brick Options Reconfigured: cluster.rebal-throttle: lazy performance.stat-prefetch: off storage.batch-fsync-delay-usec: 0 server.allow-insecure: on performance.readdir-ahead: on transport.address-family: inet nfs.disable: on
[root@node01 ~]# ls /bricks/thinvol1/brick/ | wc -w 100