分佈式文件系統-FastDFS

原創 趙大鑫 2018-09-11 07:25


一、FastDFS簡介

一、FastDFS簡介

FastDFS是由國人餘慶所開發，其項目地址：https://github.com/happyfish100
 FastDFS是一個輕量級的開源分佈式文件系統，主要解決了大容量的文件存儲和高併發訪問的問題，文件存取時實現了負載均衡。
 支持存儲服務器在線擴容,支持相同的文件只保存一份,節約磁盤。
 FastDFS只能通過Client API訪問，不支持POSIX訪問方式。
 FastDFS適合中大型網站使用，用來存儲資源文件(如：圖片、文檔、視頻等)

二、FastDFS組成部分及其它名詞

1、tracker server

跟蹤服務器：用來調度來自客戶端的請求。且在內存中記錄所有存儲組和存儲服務器的信息狀態。

2、storage server

存儲服務器：用來存儲文件(data)和文件屬性(metadata)

3、client

客戶端：業務請求發起方，通過專用接口基於TCP協議與tracker以及storage server進行交互

group

組，也可稱爲卷：同組內上的文件是完全相同的

文件標識

包括兩部分：組名和文件名(包含路徑)

meta data

文件相關屬性：鍵值對(Key Value Pair)方式

fid

文件標識符：
例如：
group1/M00/00/00/CgEOxVegXB2AdYafAAAB0b8tBbQ9155303
group_name：存儲組的組名；上傳完成後，需要客戶端自行保存
M##：服務器配置的虛擬路徑，與磁盤選項store_path#對應
兩級以兩位16進制數字命名的目錄
文件名：與原文件名並不相同；由storage server根據特定信息生成。文件名包含：源存儲服務器的IP地址、文件創建時間戳、文件大小、隨機數和文件擴展名等

三、FastDFS同步機制

1、同一組內的storage server之間是對等的，文件上傳、刪除等操作可以在任意一臺storage server上進行；
 2、文件同步只在同組內的storage server之間進行，採用push方式，即源服務器同步給目標服務器；
 3、源頭數據才需要同步，備份數據不需要再次同步，否則就構成環路了；
 上述第二條規則有個例外，就是新增加一臺storage server時，由已有的一臺storage server將已有的所有數據（包括源頭數據和備份數據）同步給該新增服務器。

四、FastDFS與集中存儲方式對比

fastdfs2.png

五、FastDFS和mogileFS對比

fastdfs1.png

六、FastDFS安裝配置

1、安裝開發環境

# yum groupinstall "Development Tools" "Server platform Development"

2、安裝libfastcommon

# git clone https://github.com/happyfish100/libfastcommon.git
# cd libfastcommon/
# ./make.sh 
# ./make.sh install

3、安裝fastdfs

# cd /root/
# git clone https://github.com/happyfish100/fastdfs.git
# cd fastdfs/
# ./make.sh 
# ./make.sh install

tracker 配置

配置文件修改：根據需求修改

# cd /etc/fdfs
# cp tracker.conf.sample tracker.conf
# vim /etc/fdfs/tracker.conf
  base_path=/data/fdfs/tracker # 存儲日誌及tracker的狀態信息

啓動服務（Centos6）：

# mkdir -pv /data/fdfs/tracker
# service fdfs_trackerd start

啓動服務（Centos7）方式一：

# mkdir -pv /data/fdfs/tracker
# /etc/init.d/fdfs_trackerd start

啓動服務（Centos7）方式二：
添加systemd的units文件

# cat > /usr/lib/systemd/system/fdfs_trackerd << EOF
# Systemd unit file for default tomcat
# 

[Unit]
Description=FastDFS tracker script
After=syslog.target network.target

[Service]
Type=notify
ExecStart=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf


[Install]
WantedBy=multi-user.target
EOF

通過systemd啓動

systemctl start fdfs_trackerd.service
# ss -tnl|grep 22122
LISTEN     0      128          *:22122                    *:*  

storage 配置

根據需求修改

# cd /etc/fdfs
# cp storage.conf.sample storage.conf
# vim /etc/fdfs/storage.conf
  group_name=group1  #指定組名
  base_path=/data/fdfs/storage # 用於存儲數據
  store_path_count=2 # 設置設備數量
  store_path0=/data/fdfs/storage/m0 #指定存儲路徑0
  store_path1=/data/fdfs/storage/m1 #指定存儲路徑1
  # 注意：同一組內存儲路徑不能衝突，例如：下一個節點的存儲路徑就是m2,m3....等
  tracker_server=10.1.14.197:22122 #指定tracker

啓動服務（Centos6）：

# mkdir -pv /data/fdfs/storage/{m0,m1} # 創建數據目錄
# service fdfs_storaged start

啓動服務（Centos7）方式一：

# mkdir -pv /data/fdfs/storage/{m0,m1} # 創建數據目錄
# /etc/init.d/fdfs_storaged start

啓動服務（Centos7）方式二：
添加systemd的units文件

# mkdir -pv /data/fdfs/storage/{m0,m1} # 創建數據目錄

# cat > /usr/lib/systemd/system/fdfs_trackerd << EOF
# Systemd unit file for default fastdfs storage
# 

[Unit]
Description=FastDFS storage script
After=syslog.target network.target

[Service]
Type=notify
ExecStart=/usr/bin/fdfs_storaged /etc/fdfs/storage.conf


[Install]
WantedBy=multi-user.target
EOF

通過systemd啓動

systemctl start fdfs_storaged.service
# ss -tnl|grep 23000
LISTEN     0      128          *:23000                    *:*  

client配置

修改客戶端配置文件

base_path=/data/fdfs/client
tracker_server=10.1.14.197:22122

七、測試FastDFS

1、上傳文件

fdfs_upload_file   [storage_ip:port] [store_path_index]

# fdfs_upload_file /etc/fdfs/client.conf /etc/issue
group1/M00/00/00/CgEOxVeMxeuABKiEAAAAF30Ccq85795930

2、查看文件

fdfs_file_info

# fdfs_file_info /etc/fdfs/client.conf group1/M00/00/00/CgEOxVeMxeuABKiEAAAAF30Ccq85795930

source storage id: 0
source ip address: 10.1.14.198
file create timestamp: 2016-07-18 20:04:59
file size: 23
file crc32: 2097312431 (0x7D0272AF)

3、下載文件

fdfs_download_file   [local_filename] [ ]

# fdfs_download_file /etc/fdfs/client.conf group1/M00/00/00/CgEOxVeMxeuABKiEAAAAF30Ccq85795930 /tmp/issue

4、查看存儲節點狀態

# fdfs_monitor /etc/fdfs/client.conf
[2016-08-03 11:58:20] DEBUG - base_path=/data/fdfs/client, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0

server_count=1, server_index=0

tracker server is 10.1.14.197:22122

group count: 1

Group 1:
group name = mage1
disk total space = 18898 MB
disk free space = 17222 MB
trunk free space = 0 MB
storage server count = 2
active server count = 2
storage server port = 23000
storage HTTP port = 8888
store path count = 1
subdir count per path = 256
current write server index = 1
current trunk file id = 0

    Storage 1:
        id = 10.1.14.198
        ip_addr = 10.1.14.198  ACTIVE
        http domain = 
        version = 5.08
        join time = 2016-08-03 20:40:46
        up time = 2016-08-03 11:58:16
        total storage = 18898 MB
        free storage = 17222 MB
        upload priority = 10
        store_path_count = 1
        subdir_count_per_path = 256
        storage_port = 23000
        storage_http_port = 8888
        current_write_path = 0
        source storage id = 
        if_trunk_server = 0
        connection.alloc_count = 256
        connection.current_count = 0
        connection.max_count = 0
        total_upload_count = 1
        success_upload_count = 1
        total_append_count = 0
        success_append_count = 0
        total_modify_count = 0
        success_modify_count = 0
        total_truncate_count = 0
        success_truncate_count = 0
        total_set_meta_count = 0
        success_set_meta_count = 0
        total_delete_count = 0
        success_delete_count = 0
        total_download_count = 0
        success_download_count = 0
        total_get_meta_count = 0
        success_get_meta_count = 0
        total_create_link_count = 0
        success_create_link_count = 0
        total_delete_link_count = 0
        success_delete_link_count = 0
        total_upload_bytes = 23
        success_upload_bytes = 23
        total_append_bytes = 0
        success_append_bytes = 0
        total_modify_bytes = 0
        success_modify_bytes = 0
        stotal_download_bytes = 0
        success_download_bytes = 0
        total_sync_in_bytes = 0
        success_sync_in_bytes = 0
        total_sync_out_bytes = 0
        success_sync_out_bytes = 0
        total_file_open_count = 1
        success_file_open_count = 1
        total_file_read_count = 0
        success_file_read_count = 0
        total_file_write_count = 1
        success_file_write_count = 1
        last_heart_beat_time = 2016-08-03 11:58:15
        last_source_update = 2016-08-03 20:43:55
        last_sync_update = 1970-01-01 08:00:00
        last_synced_timestamp = 1970-01-01 08:00:00 
    Storage 2:
        id = 10.1.14.199
        ip_addr = 10.1.14.199  ACTIVE
        http domain = 
        version = 5.08
        join time = 2016-08-03 20:42:13
        up time = 2016-08-03 11:58:08
        total storage = 18898 MB
        free storage = 17222 MB
        upload priority = 10
        store_path_count = 1
        subdir_count_per_path = 256
        storage_port = 23000
        storage_http_port = 8888
        current_write_path = 0
        source storage id = 10.1.14.198
        if_trunk_server = 0
        connection.alloc_count = 256
        connection.current_count = 0
        connection.max_count = 0
        total_upload_count = 0
        success_upload_count = 0
        total_append_count = 0
        success_append_count = 0
        total_modify_count = 0
        success_modify_count = 0
        total_truncate_count = 0
        success_truncate_count = 0
        total_set_meta_count = 0
        success_set_meta_count = 0
        total_delete_count = 0
        success_delete_count = 0
        total_download_count = 0
        success_download_count = 0
        total_get_meta_count = 0
        success_get_meta_count = 0
        total_create_link_count = 0
        success_create_link_count = 0
        total_delete_link_count = 0
        success_delete_link_count = 0
        total_upload_bytes = 0
        success_upload_bytes = 0
        total_append_bytes = 0
        success_append_bytes = 0
        total_modify_bytes = 0
        success_modify_bytes = 0
        stotal_download_bytes = 0
        success_download_bytes = 0
        total_sync_in_bytes = 23
        success_sync_in_bytes = 23
        total_sync_out_bytes = 0
        success_sync_out_bytes = 0
        total_file_open_count = 1
        success_file_open_count = 1
        total_file_read_count = 0
        success_file_read_count = 0
        total_file_write_count = 1
        success_file_write_count = 1
        last_heart_beat_time = 2016-08-03 11:58:07
        last_source_update = 1970-01-01 08:00:00
        last_sync_update = 2016-08-03 20:43:57
        last_synced_timestamp = 2016-08-03 20:43:55 (0s delay)

八、配置nginx爲storage server提供http訪問接口

1、下載fastdfs-nginx-module

# git clone https://github.com/happyfish100/fastdfs-nginx-module.git

2、下載nginx源碼，並編譯支持fastdfs

# 安裝依賴程序
# yum install openssl-devel pcre-devel -y &>/dev/null

# wget http://nginx.org/download/nginx-1.10.1.tar.gz &>/dev/null

# tar xf /root/nginx-1.10.1.tar.gz

# cd /root/nginx-1.10.1
# useradd -r nginx &>/dev/null

# ./configure --prefix=/usr/local/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx/nginx.pid --lock-path=/var/lock/nginx.lock --user=nginx --group=nginx --with-http_ssl_module --with-http_stub_status_module --with-pcre --add-module=../fastdfs-nginx-module/src
# make
# make install

3、複製配置文件

# cp /root/fastdfs-nginx-module/src/mod_fastdfs.conf  /etc/fdfs/
# cp /root/fastdfs-5.0.8/conf/{http.conf,mime.types}  /etc/fdfs/

注意：fastdfs-5.0.8爲fastdfs源碼目錄，如爲更改，應叫fastdfs

4、配置fastdfs-nginx-module配置文件

# vim /etc/fdfs/mod_fastdfs.conf
base_path=/data/fdfs/storage #存儲節點的目錄位置
tracker_server=10.1.14.197:22122 #制定tracker-server
storage_server_port=23000
group_name=mage1  #制定組名
url_have_group_name = true  #訪問路徑中是否包括組名
store_path_count=1 #配置路徑個數
store_path0=/data/fdfs/storage/m0  #指定要查看的路徑

[group1]
group_name=mage1
storage_server_port=23000
store_path_count=1
store_path0=/data/fdfs/storage/m0

5、配置nginx

# vim /etc/nginx/nginx.conf
location ~ /mage[0-9]+/M00/ {
    root /data/fdfs/storage/m0/data/;
    ngx_fastdfs_module;
}       
# cat >> /etc/profile.d/nginx.sh << EOF
export PATH=$PATH:/usr/local/nginx/sbin
EOF

# source /etc/profile.d/nginx.sh

6、爲存儲文件路徑穿件鏈接至M00

# ln -sv /data/fdfs/storage/m0/data  /data/fdfs/storage/m0/data/M00

7、啓動nginx和重啓storage並上傳文件測試

啓動nginx
# nginx -t
# nginx
# /etc/init.d/fdfs_storaged restart
# ss -tnl|grep -E "(80|23000)"
LISTEN     0      128          *:80                       *:*                  
LISTEN     0      128          *:23000                    *:*  

上傳文件

# fdfs_upload_file /etc/fdfs/client.conf /usr/share/wallpapers/CentOS7/contents/images/2560x1600.jpg
mage1/M00/00/00/CgEOx1ehc2eATt6RAA6q2wjnW8s035.jpg

查看測試

fastdfs3.png

 作者：Antony WX&QQ：1257465991 
 Q/A：如有問題請慷慨提出

如有格式問題請訪問：
http://zantony.top/2016/08/02/%E5%88%86%E5%B8%83%E5%BC%8F%E6%96%87%E4%BB%B6%E7%B3%BB%E7%BB%9F-FastDFS/

發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

bat記錄遠程桌面連接登錄信息

jason1982 2019-02-24 12:57:09

Exchange Server 2010 POP3&IMAP設置詳解

lingping 2019-02-23 14:05:54

虛擬化技術--服務器虛擬化

dawei818 2019-02-23 14:05:39

windows 2008 全新仲裁模式

qyh282110204 2019-02-23 14:05:36

軟路由建立PPPOE服務器

ct19871125 2019-02-23 14:04:24

MySQL性能優化的21個最佳實踐

fdb2b 2019-02-23 14:01:03

centos下簡單實現日誌切割，並上傳至日誌服務器。

隨風上升 2019-02-23 13:59:02

遠程訪問及控制

曉晶 2019-02-23 13:58:40

web網站服務1

曉晶 2019-02-23 13:58:27

郵件服務器

曉晶 2019-02-23 13:58:27

web網站服務2

曉晶 2019-02-23 13:58:27

iscsi存儲

samplelife 2019-02-23 13:57:35

Nagios 3.2 監控部署(一)

samplelife 2019-02-23 13:57:35

[轉]SMTP&郵件系統常見錯誤代碼

lingping 2019-02-23 14:05:54

tortoisegit push 出錯 沒有生成puttykey

lftong 2019-02-23 14:00:30