環境：centos8虛擬機（centos-release-8.1-1.1911）
方法：
- qemu-img、qemu-nbd創建磁盤，關於nbd可以參考文章linux qemu-nbd介紹
- 使用一個虛擬機來模擬兩個節點
1. 安裝軟件包
- 安裝glusterfs源
# yum install centos-release-gluster7
得到/etc/yum.repos.d/CentOS-Gluster-7.repo
- 重新創建yum源cache
# make makecache
- 安裝glusterfs
# yum install -y glusterfs glusterfs-server
2. 配置glusterfs
- 啓動服務
# systemctl start glusterd
- 配置域名解析
# echo "169.254.155.112 build-centos81" >> /etc/hosts
# hostname
build-centos81
# hostname -i
169.254.155.112
- 配置域名可信任
# gluster peer probe build-centos81
由於只有一個節點，這個可以不配置。多個節點需要相互配置可信任域名
3. 使用qemu-img創建磁盤
- 創建一個1G磁盤並掛載
# qemu-img create -f qcow2 /home/gluster1.qcow2 1G // qemu-img創建1G磁盤
# qemu-nbd -c /dev/nbd0 /homt/gluster1.qcow2 // qemu-nbd掛載
# mkfs.ext4 /dev/nbd0 // 格式化磁盤
# mkdir -p /home/data1/brick1 // 創建掛載點
# mount /dev/nbd0 /home/data1/brick1 // 掛載磁盤
# mkdir -p /home/data1/brick1/gv0 // 創建gv0目錄，後面創建glusterfs會用到
- 創建一個2G磁盤並掛載，過程同上
# qemu-img create -f qcow2 /home/gluster2.qcow2 2G // qemu-img創建2G磁盤
# qemu-nbd -c /dev/nbd1 /homt/gluster2.qcow2 // qemu-nbd掛載
# mkfs.ext4 /dev/nbd1 // 格式化磁盤
# mkdir -p /home/data2/brick1 // 創建掛載點
# mount /dev/nbd1 /home/data2/brick1 // 掛載磁盤
# mkdir -p /home/data2/brick1/gv0 // 創建gv0目錄，後面創建glusterfs會用到
4. 創建glusterfs文件系統
- 創建volume
# gluster volume create gv0 replica 2 build-centos81:/home/data1/brick1/gv0 build-centos81:/home/data2/brick1/gv0 force
因爲我們是在一個系統上模擬兩個節點，這裏要使用force參數，否則會出現下面的錯誤信息：
volume create: gv0: failed: Multiple bricks of a replicate volume are present on the same server. This setup is not optimal. Bricks should be on different nodes to have best fault tolerant configuration. Use ‘force’ at the end of the command if you want to override this behavior.
- 啓動volume
# gluster volume start gv0
- 查看創建的volume信息
# gluster volume info
Volume Name: gv0
Type: Replicate
Volume ID: 7bf1b0e2-1dd2-44fe-a209-bc5ca430a350
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: build-centos81:/home/data1/brick1/gv0
Brick2: build-centos81:/home/data2/brick1/gv0
Options Reconfigured:
transport.address-family: inet
storage.fips-mode-rchecksum: on
nfs.disable: on
performance.client-io-threads: off
- 掛載glusterfs
# mount.glusterfs build-centos81:/gv0 /mnt
- 查看大小
# df -h | grep mnt
build-centos81:/gv0 976M 13M 907M 2% /mnt
可以看到掛載後的空間大小等於兩個節點中較小的一個
5. 擴容
以上創建的兩個節點的glusterfs，因此每次擴容需要兩個節點。創建兩個磁盤，每個磁盤爲2G。
- 擴容
# gluster volume add-brick gv0 build-centos81:/home/data3/brick1/gv0 build-centos81:/home/data4/brick1/gv0
# gluster volume info
Volume Name: gv0
Type: Distributed-Replicate
Volume ID: 7bf1b0e2-1dd2-44fe-a209-bc5ca430a350
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: build-centos81:/home/data1/brick1/gv0
Brick2: build-centos81:/home/data2/brick1/gv0
Brick3: build-centos81:/home/data3/brick1/gv0
Brick4: build-centos81:/home/data4/brick1/gv0
Options Reconfigured:
transport.address-family: inet
storage.fips-mode-rchecksum: on
nfs.disable: on
performance.client-io-threads: off
# df -h | grep mnt
build-centos81:/gv0 2.9G 38M 2.7G 2% /mnt
6. 縮容
- 縮容
# gluster volume remove-brick gv0 build-centos81:/home/data3/brick1/gv0 build-centos81:/home/data4/brick1/gv0 start
Do you want to continue with your current cluster.force-migration settings? (y/n) y
volume remove-brick start: success
ID: 59a319e7-f055-4471-be3c-19c83b89665b
# gluster volume remove-brick gv0 build-centos81:/home/data3/brick1/gv0 build-centos81:/home/data4/brick1/gv0 status
Node Rebalanced-files size scanned failures skipped status run time in h:m:s
--------- ----------- ----------- ----------- ----------- ----------- ------------ --------------
localhost 0 0Bytes 1 0 0 completed 0:00:00
# gluster volume remove-brick gv0 build-centos81:/home/data3/brick1/gv0 build-centos81:/home/data4/brick1/gv0 commit
volume remove-brick commit: success
7. 刪除glusterfs
- umount掛載點
# umount /mnt
- 停止glusterfs
# gluster volume stop gv0
- 刪除glusterfs
# gluster volume delete gv0
# gluster volume info
No volumes present