目錄
軟件的安裝、配置
安裝列表
fastdfs的安裝需要centos，我在ubuntu上安裝沒有成功
安裝所需的依賴包
安裝前可使用dpkg -l | grep gcc命令來查看依賴包是否已經安裝，如果沒有，使用下列命令來安裝相應的依賴包
yum -y install libevent //Libevent 是一個用C語言編寫的、輕量級的開源高性能事件通知庫
yum -y install pop* //安裝popler,popt等庫
yum -y install openssl* //使得nginx不僅支持http，還支持https
yum install -y gcc gcc-c++ //安裝gcc，用於編譯nginx
安裝libfastcommon
將libfastcommon-1.0.7.tar.gz解壓至指定目錄（所有安裝包都在/home/sword/distributed-storage/），然後使用以下命令來安裝。
./make.sh
./make.sh install
libfastcommon安裝好後會自動將庫文件拷貝至/usr/lib64下，由於FastDFS程序引用usr/lib目錄所以需要將/usr/lib64下的庫文件拷貝至/usr/lib下
cp /usr/lib64/libfastcommon.so /usr/lib/
安裝FastDFS
將FastDFS_v5.05.tar.gz解壓到指定目錄，然後使用以下命令來安裝
./make.sh
./make.sh install
安裝完成後在/etc/fdfs下，將三個文件：
client.conf.sample
storage.conf.sample
tracker.conf.sample
都修改爲：
client.conf
storage.conf
tracker.conf
配置tracker服務器，並啓動
修改/etc/fdfs/tracker.conf
//下面這個路徑需要自己新建
base_path=/home/sword/Desktop/distributed-storage/storage-data
http.server_port=8080
配置完成後使用下面命令行啓動tracker服務
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
使用下面的命令查看是否啓動成功
ps -e|grep fdfs_trackerd
如果成功，則會有
[root@localhost bin]# ps -e|grep fdfs_trackerd
15425 ? 00:00:00 fdfs_trackerd
配置Storage服務器，並啓動
修改/etc/fdfs/tracker.conf
group_name=group1
base_path=/home/sword/Desktop/distributed-storage/storage-data
store_path0=/home/sword/Desktop/distributed-storage/storage-data
tracker_server=192.168.217.144:22122
//如果有多個則配置多個tracker
//tracker_server=192.168.217.145:22122
配置完成後使用下面命令啓動Storage服務
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
使用下面的命令查看tracker和storage是否都啓動成功
[root@localhost bin]# ps -e|grep fdfs_
15425 ? 00:00:00 fdfs_trackerd
24134 pts/0 00:00:00 fdfs_storaged
修改Client，以測試圖片上傳
修改/etc/fdfs/client.conf
base_path=/home/sword/Desktop/distributed-storage/storage-data
tracker_server=192.168.217.144:22122
//tracker_server=192.168.217.145:22122
使用以下命令上傳圖片
[root@localhost distributed-storage]# /usr/bin/fdfs_test /etc/fdfs/client.conf upload 1.jpg
結果如下
This is FastDFS client test program v5.05
Copyright (C) 2008, Happy Fish / YuQing
FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/
for more detail.
[2020-01-05 02:58:15] DEBUG - base_path=/home/sword/Desktop/distributed-storage/storage-data, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
tracker_query_storage_store_list_without_group:
server 1. group_name=, ip_addr=192.168.217.144, port=23000
group_name=group1, ip_addr=192.168.217.144, port=23000
storage_upload_by_filename
group_name=group1, remote_filename=M00/00/00/wKjZkF4Q4EeANDdQAAGNecD0vxY622.jpg
source ip address: 192.168.217.144
file timestamp=2020-01-05 02:58:15
file size=101753
file crc32=3237265174
example file url: http://192.168.217.144/group1/M00/00/00/wKjZkF4Q4EeANDdQAAGNecD0vxY622.jpg
storage_upload_slave_by_filename
group_name=group1, remote_filename=M00/00/00/wKjZkF4Q4EeANDdQAAGNecD0vxY622_big.jpg
source ip address: 192.168.217.144
file timestamp=2020-01-05 02:58:15
file size=101753
file crc32=3237265174
example file url: http://192.168.217.144/group1/M00/00/00/wKjZkF4Q4EeANDdQAAGNecD0vxY622_big.jpg
FDFS和Nginx組合
FastDFS的HTTP服務較爲簡單，無法提供負載均衡等高性能的服務，所以FastDFS的作者爲我們提供了Nginx上使用的FastDFS模塊。
新建一個nginx的安裝目錄（nginx_LJ），將fastdfs-nginx-module，nginx-1.9.3，ngx_cache_purge-2.3，pcre-8.37，zlib-1.2.3解壓，目前的目錄結構是
修改/home/sword/Desktop/distributed-storage/fastdfs-nginx-module/src/中的config文件（好像不需要改，在下一步建立軟連接就可以了，不改的話，make的時候會出現找不到fdfs_define.h的錯誤）
ngx_addon_name=ngx_http_fastdfs_module
HTTP_MODULES="$HTTP_MODULES ngx_http_fastdfs_module"
NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_fastdfs_module.c"
CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/"
CORE_LIBS="$CORE_LIBS -L/usr/local/lib -lfastcommon -lfdfsclient"
CFLAGS="$CFLAGS -D_FILE_OFFSET_BITS=64 -DFDFS_OUTPUT_CHUNK_SIZE='256*1024' -DFDFS_MOD_CONF_FILENAME='\"/etc/fdfs/mod_fastdfs.conf\"'"
使用三個命令分別建立三個軟連接
ln -sv /usr/include/fastcommon /usr/local/include/fastcommon
ln -sv /usr/include/fastdfs /usr/local/include/fastdfs
ln -sv /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
然後使用下面命令進行安裝
./configure --prefix=/home/sword/Desktop/distributed-storage/nginx_LJ --add-module=/home/sword/Desktop/distributed-storage/fastdfs-nginx-module/src --add-module=/home/sword/Desktop/distributed-storage/ngx_cache_purge-2.3 --with-pcre=/home/sword/Desktop/distributed-storage/pcre-8.37 --with-zlib=/home/sword/Desktop/distributed-storage/zlib-1.2.3
make&make install
使用Nginx來訪問FastDFS中的文件
修改nginx.conf中的用戶，否則在訪問的時候會出現無法連接的情況。找到設置user的地方（文件的開頭位置）
user root;
添加location
server {
listen 8081;
server_name localhost;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root html;
index index.html index.htm;
}
location /group1/M00 {
root /home/sword/Desktop/distributed-storage/storage-data/data;
ngx_fastdfs_module;
}
在/home/sword/Desktop/distributed-storage/FastDFS/conf/找到http.conf和mime.types，拷貝到/etc/fdfs目錄下
cp http.conf /etc/fdfs/
cp mime.types /etc/fdfs/
在/home/sword/Desktop/distributed-storage/fastdfs-nginx-module/src/中找到mod_fastdfs.conf，拷貝到/etc/fdfs目錄下
cp mod_fastdfs.conf /etc/fdfs/
修改這個文件的內容如下：
# connect timeout in seconds
# default value is 30s
connect_timeout=2
# network recv and send timeout in seconds
# default value is 30s
network_timeout=30
# the base path to store log files
base_path=/home/sword/Desktop/distributed-storage/
# if load FastDFS parameters from tracker server
# since V1.12
# default value is false
load_fdfs_parameters_from_tracker=true
# storage sync file max delay seconds
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# since V1.12
# default value is 86400 seconds (one day)
storage_sync_file_max_delay = 86400
# if use storage ID instead of IP address
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# default value is false
# since V1.13
use_storage_id = false
# specify storage ids filename, can use relative or absolute path
# same as tracker.conf
# valid only when load_fdfs_parameters_from_tracker is false
# since V1.13
storage_ids_filename = storage_ids.conf
# FastDFS tracker_server can ocur more than once, and tracker_server format is
# "host:port", host can be hostname or ip address
# valid only when load_fdfs_parameters_from_tracker is true
tracker_server=192.168.217.144:22122
# the port of the local storage server
# the default value is 23000
storage_server_port=23000
# the group name of the local storage server
group_name=group1
# if the url / uri including the group name
# set to false when uri like /M00/00/00/xxx
# set to true when uri like ${group_name}/M00/00/00/xxx, such as group1/M00/xxx
# default value is false
url_have_group_name = true
# path(disk or mount point) count, default value is 1
# must same as storage.conf
store_path_count=1
# store_path#, based 0, if store_path0 not exists, it's value is base_path
# the paths must be exist
# must same as storage.conf
store_path0=/home/sword/Desktop/distributed-storage/storage-data
#store_path1=/home/yuqing/fastdfs1
# standard log level as syslog, case insensitive, value list:
### emerg for emergency
### alert
### crit for critical
### error
### warn for warning
### notice
### info
### debug
log_level=info
# set the log filename, such as /usr/local/apache2/logs/mod_fastdfs.log
# empty for output to stderr (apache and nginx error_log file)
log_filename=
# response mode when the file not exist in the local file system
## proxy: get the content from other storage server, then send to client
## redirect: redirect to the original storage server (HTTP Header is Location)
response_mode=proxy
# the NIC alias prefix, such as eth in Linux, you can see it by ifconfig -a
# multi aliases split by comma. empty value means auto set by OS type
# this paramter used to get all ip address of the local host
# default values is empty
if_alias_prefix=
# use "#include" directive to include HTTP config file
# NOTE: #include is an include directive, do NOT remove the # before include
#include http.conf
# if support flv
# default value is false
# since v1.15
flv_support = true
# flv file extension name
# default value is flv
# since v1.15
flv_extension = flv
# set the group count
# set to none zero to support multi-group
# set to 0 for single group only
# groups settings section as [group1], [group2], ..., [groupN]
# default value is 0
# since v1.14
group_count = 0
# group settings for group #1
# since v1.14
# when support multi-group, uncomment following section
[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/home/sword/Desktop/distributed-storage/storage-data
# group settings for group #2
# since v1.14
# when support multi-group, uncomment following section as neccessary
#[group2]
#group_name=group2
#storage_server_port=23000
#store_path_count=2
#store_path0=/home/yuqing/fastdfs
#store_path1=/home/yuqing/fastdfs1
創建M00至storage存儲目錄的符號連接：
ln -s /home/sword/Desktop/distributed-storage/storage-data/data/ /home/sword/Desktop/distributed-storage/storage-data/data/M00
在/home/sword/Desktop/distributed-storage/storage-data/data/sync/目錄下找到binlog.000文件，查看已經上傳的文件
[root@localhost sync]# cat binlog.000
1578164295 C M00/00/00/wKjZkF4Q4EeANDdQAAGNecD0vxY622.jpg
1578164294 C M00/00/00/wKjZkF4Q4EeANDdQAAGNecD0vxY622.jpg-m
1578164295 C M00/00/00/wKjZkF4Q4EeANDdQAAGNecD0vxY622_big.jpg
1578164295 C M00/00/00/wKjZkF4Q4EeANDdQAAGNecD0vxY622_big.jpg-m
打開/home/sword/Desktop/distributed-storage/nginx_LJ/sbin/，啓動nginx
./nginx
在瀏覽器中瀏覽nginx主頁
通過nginx訪問fdfs上的內容
程序操作FDFS
Python
from fdfs_client.client import *
import time
class FDFSClient:
def __init__(self,config_file_path="client.conf"):
self.config_file_path = config_file_path
self.tracker = get_tracker_conf(self.config_file_path)
self.client = Fdfs_client(self.tracker)
#上傳文件字節
def upload_file_by_buffer(self,file_path=""):
with open(file_path,"rb") as f:
ret = self.client.upload_by_buffer(f.read())
print(ret)
return ret
#上傳文件
def upload_file(self,file_path=""):
ret = self.client.upload_by_filename(file_path)
print(ret)
return ret
#下載文件字節，是不是可以實現以流的方式播放，有待驗證
def download_file_by_buffer(self,local_file="",remote_file_id=b"",offset=0,down_bytes=1024):
ret = self.client.download_to_buffer(remote_file_id,offset=offset,down_bytes=down_bytes)
with open(local_file,"wb") as f:
f.write(ret["Content"])
def download_file(self,local_file="",remote_file_id=b"",offset=0,down_bytes=1024):
ret = self.client.download_to_file(local_file,remote_file_id)
def delete_file(self,remote_file_id):
ret = self.client.delete_file(remote_file_id)
print(ret)
if __name__=="__main__":
fdfsClient = FDFSClient()
#ret = fdfsClient.upload_file("1.MP4")
#ret = fdfsClient.upload_file_by_buffer("1.MP4")
#remote_file_id = ret["Remote file_id"]
#fdfsClient.download_file_by_buffer("test.MP4",remote_file_id=b'group1/M00/00/00/wKjZkF4TXKiAPiXhBx77kEARHrg3654480')
fdfsClient.download_file("test.MP4",remote_file_id=b'group1/M00/00/00/wKjZkF4TZLiAY9ReBx77kEARHrg163.MP4')
#fdfsClient.delete_file(b'group1/M00/00/00/wKjZkF4TXKiAPiXhBx77kEARHrg3654480')