SpringBoot 實現大文件斷點續傳(前端基於WebUploader實現,1G文件上傳只需5s)

最近在工作中有涉及到文件上傳功能,需求方要求文件最大上限爲2G,此時如果直接將文件在前端做上傳,會出現超長時間等待,如果服務端內存不夠,會直接內存溢出,此時我們可以通過斷點續傳方式解決,前端我們通過WebUploader實現文件分割和上傳,後端我們通過SpringBoot實現文件接收和組裝功能,下面我列出前後端主要功能代碼(前後端完整代碼見文末下載地址)。

1、前端代碼

我們在前端代碼中註冊3個事件,分別是 before-send-file、before-send、after-send-file,這三個鉤子分別是在發送文件前(上傳文件之前執行,觸發一次)、發送請求前(上傳文件分塊之前執行,觸發多次)、文件上傳後(分塊全部上傳完成之後執行,觸發一次)

html代碼如下:

<input
  disabled={inputDisabled}
  type="file"
  title=""
  id="fileupload"
  accept=".png, .jpg, .jpeg"
  multiple={false}
  name="file"
  onClick={e => (e.target.value = '')}
  onChange={e => inputOnChange(e)}
/>

點擊事件代碼如下:

const inputOnChange = e => {
  e.preventDefault();
  const files = e.target.files || e.dataTransfer.files;
  if (files && files[0]) {
    startUpload(files[0]);
  }
}

const startUpload = targetFile => {
  const uploader = new Uploader({
    file: targetFile,
    onSuccess: ({ fileName, resourceId, filePath }) => {
      // ...
    },
    onError: ({ msg }) => {
      // ...
    },
    onProgress: ({ data, percentage }) => {
      // ...
    },
  });

  uploader.start();
}

WebUploader斷點續傳代碼如下:

import request from '@/utils/request';
import WebUploader from '../../public/webuploader.min';
import { TP_TOKE, BPR_BASE_URL } from '@/utils/constant';

/**
 *
 * 斷點續傳純邏輯組件
 *
 * 用法:
 * ```
 * uploader = new Uploader({
 *     file: targetFile,
 *     onSuccess: ({ fileName, resourceId, filePath }) => {
 *     },
 *     onError: ({ msg }) => {
 *     },
 *     onProgress: ({ data, percentage }) => {
 *     },
 *   });
 *
 * uploader.start();
 * ```
 * @class Uploader
 */
class Uploader {
  constructor({ file, onSuccess, onError, onProgress }) {
    // const files = e.target.files || e.dataTransfer.files;
    // 轉化爲WebUploader的內部file對象
    this.file = new WebUploader.File(new WebUploader.Lib.File(WebUploader.guid('rt_'), file));
    this.onSuccess = props => {
      this.clean();
      if (onSuccess) onSuccess(props);
    };
    this.onError = props => {
      this.clean();
      if (onError) onError(props);
    };
    this.onProgress = onProgress;
    this.uploader = null;
  }

  init = () => {
    WebUploader.Uploader.register({
      name: 'webUploaderHookCommand',
      'before-send-file': 'beforeSendFile',
      'before-send': 'beforeSend',
      'after-send-file': 'afterSendFile',
    }, {
      beforeSendFile: file => {
        const task = new WebUploader.Deferred();
        this.fileName = file.name;
        this.fileSize = file.size;
        this.mimetype = file.type;
        this.fileExt = file.ext;
        (new WebUploader.Uploader())
          .md5File(file, 0, 10 * 1024 * 1024 * 1024 * 1024).progress(percentage => { })
          .then(val => {
            this.fileMd5 = val;
            const url = `${BPR_BASE_URL}/register`;
            const data = {
              fileMd5: this.fileMd5,
              fileName: file.name,
              fileSize: file.size,
              mimetype: file.type,
              fileExt: file.ext,
            };
            request(url, {
              method: 'post',
              data,
            }).then(res => {
              console.log('register', res);
              if (res && res.status === 1) {
                task.resolve();
              } else if (res && res.data && res.code === 103404) {
                // 文件已上傳
                this.onSuccess({
                  fileName: this.fileName,
                  resourceId: res.data.resId,
                  filePath: res.data.filePath,
                });
                task.reject();
              } else {
                file.statusText = res && res.message;
                task.reject();
              }
            });
          });
        return task.promise();
      },
      beforeSend: block => {
        console.log('beforeSend');
        const task = new WebUploader.Deferred();
        const url = `${BPR_BASE_URL}/checkChunk`;
        const data = {
          fileMd5: this.fileMd5,
          chunk: block.chunk,
          chunkSize: block.end - block.start,
        };
        request(url, {
          method: 'post',
          data,
        }).then(res => {
          console.log('checkChunk', res);
          if (res && res.data === true) {
            task.reject(); // 分片存在,則跳過上傳
          } else {
            task.resolve();
          }
        });
        this.uploader.options.formData.fileMd5 = this.fileMd5;
        this.uploader.options.formData.chunk = block.chunk;
        return task.promise();
      },
      afterSendFile: () => {
        console.log('start afterSendFile');
        const task = new WebUploader.Deferred();
        const url = `${BPR_BASE_URL}/mergeChunks`;
        const data = {
          fileMd5: this.fileMd5,
          fileName: this.fileName,
          fileSize: this.fileSize,
          mimetype: this.mimetype,
          fileExt: this.fileExt,
        };
        request(url, {
          method: 'post',
          data,
        }).then(res => {
          console.log('mergeChunks', res);
          if (res && res.status === 1 && res.data && res.data.resId) {
            task.resolve();
            this.onSuccess({
              fileName: this.fileName,
              resourceId: res.data.resId,
              filePath: res.data.filePath,
            });
          } else {
            task.reject();
            this.onError({ msg: '合併文件失敗' });
          }
        });
      },
    });
  }

  clean = () => {
    if (this.uploader) {
      WebUploader.Uploader.unRegister('webUploaderHookCommand');
    }
  }

  start = () => {
    if (!this.uploader) {
      this.init();
    }
    // 實例化
    this.uploader = WebUploader.create({
      server: BPR_BASE_URL,
      chunked: true,
      chunkSize: 1024 * 1024 * 5,
      chunkRetry: 1,
      threads: 3,
      duplicate: true,
      formData: { // 上傳分片的http請求中一同攜帶的數據
        appid: '1',
        token: localStorage.getItem(TP_TOKE),
        methodname: 'breakpointRenewal',
      },
    });

    // 一個分片上傳成功後,調用該方法
    this.uploader.on('uploadProgress', (data, percentage) => {
      console.log('uploadProgress');
      this.onProgress({ data, percentage });
    });

    this.uploader.on('error', err => {
      this.onError({ msg: '上傳出錯,請重試' });
    });

    this.uploader.addFiles(this.file);
    this.uploader.upload();
  }

  cancel = () => {
    console.log('call cancel');
    this.uploader.stop(true);
    this.uploader.destroy();
    console.log('getStats', this.uploader.getStats());
  }
}
export default Uploader;

到這裏前端代碼就寫完了。

2、FileController代碼

package com.openailab.oascloud.file.controller;

import com.openailab.oascloud.common.model.RequestParameter;
import com.openailab.oascloud.common.model.ResponseResult;
import com.openailab.oascloud.common.model.tcm.vo.ResourceVO;
import com.openailab.oascloud.file.api.IFileController;
import com.openailab.oascloud.file.model.LoginUserInfo;
import com.openailab.oascloud.file.service.IFileService;
import com.openailab.oascloud.file.service.IUserService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RequestPart;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;

import javax.servlet.http.HttpServletResponse;
import java.util.Optional;

/**
 * @description: 文件管理-Controller
 * @author: zhangzhixiang
 * @createDate: 2019/12/9
 * @version: 1.0
 */
@RestController
public class FileController extends BaseController implements IFileController {

    private static Logger LOG = LoggerFactory.getLogger(FileController.class);
    @Autowired
    private IFileService fileService;
    @Autowired
    private IUserService userService;

    /**
     * 斷點敘傳
     *
     * @param file
     * @param fileMd5
     * @param chunk
     * @return com.openailab.oascloud.common.model.ResponseResult
     * @author zxzhang
     * @date 2020/1/13
     */
    @Override
    public ResponseResult breakpointRenewal(@RequestPart("file") MultipartFile file,
                                            @RequestParam("fileMd5") String fileMd5,
                                            @RequestParam("chunk") Integer chunk) {
        try {
            return fileService.breakpointRenewal(file, fileMd5, chunk);
        } catch (Exception e) {
            LOG.error("********FileController->breakpointRenewal throw Exception.fileMd5:{},chunk:{}********", fileMd5, chunk, e);
        }
        return ResponseResult.fail(null);
    }

    /**
     * 斷點敘傳註冊
     *
     * @param fileMd5
     * @param fileName
     * @param fileSize
     * @param mimetype
     * @param fileExt
     * @return com.openailab.oascloud.common.model.ResponseResult
     * @author zxzhang
     * @date 2020/1/13
     */
    @Override
    public ResponseResult breakpointRegister(@RequestParam("fileMd5") String fileMd5,
                                             @RequestParam("fileName") String fileName,
                                             @RequestParam("fileSize") Long fileSize,
                                             @RequestParam("mimetype") String mimetype,
                                             @RequestParam("fileExt") String fileExt) {
        try {
            return fileService.breakpointRegister(fileMd5, fileName, fileSize, mimetype, fileExt);
        } catch (Exception e) {
            LOG.error("********FileController->breakpointRegister throw Exception.fileMd5:{},fileName:{}********", fileMd5, fileName, e);
        }
        return ResponseResult.fail(null);
    }

    /**
     * 檢查分塊是否存在
     *
     * @param fileMd5
     * @param chunk
     * @param chunkSize
     * @return com.openailab.oascloud.common.model.ResponseResult
     * @author zxzhang
     * @date 2020/1/10
     */
    @Override
    public ResponseResult checkChunk(@RequestParam("fileMd5") String fileMd5,
                                     @RequestParam("chunk") Integer chunk,
                                     @RequestParam("chunkSize") Integer chunkSize) {
        try {
            return fileService.checkChunk(fileMd5, chunk, chunkSize);
        } catch (Exception e) {
            LOG.error("********FileController->breakpointRenewal throw Exception.fileMd5:{},chunk:{}********", fileMd5, chunk, e);
        }
        return ResponseResult.fail(null);
    }

    /**
     * 合併文件塊
     *
     * @param fileMd5
     * @param fileName
     * @param fileSize
     * @param mimetype
     * @param fileExt
     * @return com.openailab.oascloud.common.model.ResponseResult
     * @author zxzhang
     * @date 2020/1/11
     */
    @Override
    public ResponseResult mergeChunks(@RequestParam("fileMd5") String fileMd5,
                                      @RequestParam("fileName") String fileName,
                                      @RequestParam("fileSize") Long fileSize,
                                      @RequestParam("mimetype") String mimetype,
                                      @RequestParam("fileExt") String fileExt,
                                      @RequestParam("token") String token) {
        try {
            LoginUserInfo user = userService.getLoginUser(token);
            return fileService.mergeChunks(fileMd5, fileName, fileSize, mimetype, fileExt, user);
        } catch (Exception e) {
            LOG.error("********FileController->breakpointRenewal throw Exception.fileMd5:{},fileName:{}********", fileMd5, fileName, e);
        }
        return ResponseResult.fail(null);
    }
}

2、IFileService代碼

package com.openailab.oascloud.file.service;

import com.openailab.oascloud.common.model.RequestParameter;
import com.openailab.oascloud.common.model.ResponseResult;
import com.openailab.oascloud.common.model.tcm.vo.ResourceVO;
import com.openailab.oascloud.file.model.LoginUserInfo;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RequestPart;
import org.springframework.web.multipart.MultipartFile;

import javax.servlet.http.HttpServletResponse;
import java.io.File;
import java.io.FileNotFoundException;
import java.util.Date;
import java.util.List;

/**
 * @description: 文件管理-Interface
 * @author: zhangzhixiang
 * @createDate: 2019/12/9
 * @version: 1.0
 */
public interface IFileService {
    /**
     * 斷點敘傳註冊
     *
     * @param fileMd5
     * @param fileName
     * @param fileSize
     * @param mimetype
     * @param fileExt
     * @return com.openailab.oascloud.common.model.ResponseResult
     * @author zxzhang
     * @date 2020/1/10
     */
    ResponseResult breakpointRegister(String fileMd5, String fileName, Long fileSize, String mimetype, String fileExt);

    /**
     * 斷點敘傳
     *
     * @param file
     * @return com.openailab.oascloud.common.model.ResponseResult
     * @author zxzhang
     * @date 2019/12/9
     */
    ResponseResult breakpointRenewal(MultipartFile file, String fileMd5, Integer chunk);

    /**
     * 檢查分塊是否存在
     *
     * @param fileMd5
     * @param chunk
     * @param chunkSize
     * @return com.openailab.oascloud.common.model.ResponseResult
     * @author zxzhang
     * @date 2020/1/10
     */
    ResponseResult checkChunk(String fileMd5, Integer chunk, Integer chunkSize);

    /**
     * 合併文件塊
     *
     * @param fileMd5
     * @param fileName
     * @param fileSize
     * @param mimetype
     * @param fileExt
     * @return com.openailab.oascloud.common.model.ResponseResult
     * @author zxzhang
     * @date 2020/1/11
     */
    ResponseResult mergeChunks(String fileMd5, String fileName, Long fileSize, String mimetype, String fileExt, LoginUserInfo user);
}

3、FileServiceImpl代碼

package com.openailab.oascloud.file.service.impl;

import com.alibaba.fastjson.JSONObject;
import com.github.pagehelper.PageInfo;
import com.google.common.collect.Maps;
import com.openailab.oascloud.common.enums.ResponseEnum;
import com.openailab.oascloud.common.model.ResponseResult;
import com.openailab.oascloud.common.model.tcm.ResourceBO;
import com.openailab.oascloud.common.model.tcm.vo.ResourceVO;
import com.openailab.oascloud.common.model.um.FileUserBO;
import com.openailab.oascloud.file.common.config.BootstrapConfig;
import com.openailab.oascloud.file.common.consts.BootstrapConst;
import com.openailab.oascloud.file.common.consts.RedisPrefixConst;
import com.openailab.oascloud.file.common.enums.ResourceTypeEnum;
import com.openailab.oascloud.file.common.enums.TranscodingStateEnum;
import com.openailab.oascloud.file.common.enums.VedioEnum;
import com.openailab.oascloud.file.common.file.ClientFactory;
import com.openailab.oascloud.file.common.file.FileClient;
import com.openailab.oascloud.file.common.helper.FileManagementHelper;
import com.openailab.oascloud.file.dao.FileDao;
import com.openailab.oascloud.file.dao.RedisDao;
import com.openailab.oascloud.file.model.LoginUserInfo;
import com.openailab.oascloud.file.service.IFileService;
import com.openailab.oascloud.file.util.*;
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.compress.utils.IOUtils;
import org.apache.commons.lang.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.BeanUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.util.ObjectUtils;
import org.springframework.web.multipart.MultipartFile;

import javax.servlet.http.HttpServletResponse;
import java.io.*;
import java.text.SimpleDateFormat;
import java.util.*;

/**
 * @description: 文件管理-service
 * @author: zhangzhixiang
 * @createDate: 2019/12/9
 * @version: 1.0
 */
@Service
public class FileServiceImpl implements IFileService {

    private final static Logger LOG = LoggerFactory.getLogger(FileServiceImpl.class);
    private static final SimpleDateFormat format = new SimpleDateFormat("yyyyMMdd");
    @Autowired
    private FileDao fileDao;
    @Autowired
    private BootstrapConfig bootstrapConfig;
    @Autowired
    private FileManagementHelper fileManagementHelper;
    @Autowired
    private PageObjUtils pageObjUtils;
    @Autowired
    private RedisDao redisDao;

    private String getUploadPath() {
        return bootstrapConfig.getFileRoot() + bootstrapConfig.getUploadDir() + "/";
    }

    private String getFileFolderPath(String fileMd5) {
        return getUploadPath() + fileMd5.substring(0, 1) + "/" + fileMd5.substring(1, 2) + "/";
    }

    private String getFilePath(String fileMd5, String fileExt) {
        return getFileFolderPath(fileMd5) + fileMd5 + "." + fileExt;
    }

    private String getFileRelativePath(String fileMd5, String fileExt) {
        return bootstrapConfig.getUploadDir() + "/" + fileMd5.substring(0, 1) + "/" + fileMd5.substring(1, 2) + "/" + fileMd5 + "." + fileExt;
    }

    private String getChunkFileFolderPath(String fileMd5) {
        return bootstrapConfig.getFileRoot() + bootstrapConfig.getBreakpointDir() + "/" + fileMd5 + "/";
    }

    @Override
    public ResponseResult breakpointRegister(String fileMd5, String fileName, Long fileSize, String mimetype, String fileExt) {
        Map<String, String> ret = Maps.newHashMap();
        // 檢查文件是否存在於磁盤
        String fileFolderPath = this.getFileFolderPath(fileMd5);
        String filePath = this.getFilePath(fileMd5, fileExt);
        File file = new File(filePath);
        boolean exists = file.exists();

        // 檢查文件是否存在於PostgreSQL中 (文件唯一標識爲 fileMd5)
        ResourceBO resourceBO = new ResourceBO();
        resourceBO.setFileMd5(fileMd5);
        resourceBO.setIsDelete(0);
        List<ResourceBO> resourceBOList = fileDao.selectResourceByCondition(resourceBO);
        if (exists && resourceBOList.size() > 0) {
            // 既存在於磁盤又存在於數據庫說明該文件存在,直接返回resId、filePath
            resourceBO = resourceBOList.get(0);
            ret.put("filePath", resourceBO.getFilePath());
            ret.put("resId", String.valueOf(resourceBO.getResourceId()));
            return ResponseResult.fail(ResponseEnum.RESPONSE_CODE_BREAKPOINT_RENEVAL_REGISTRATION_ERROR, ret);
        }

        //若磁盤中存在,但數據庫中不存在,則生成resource記錄並存入redis中
        if (resourceBOList.size() == 0) {
            // 首次斷點敘傳的文件需要創建resource新記錄並返回redId,並存入redis中
            resourceBO.setType(fileManagementHelper.judgeDocumentType(fileExt));
            resourceBO.setStatus(TranscodingStateEnum.UPLOAD_NOT_COMPLETED.getCode());
            resourceBO.setFileSize(fileSize);
            resourceBO.setFileMd5(fileMd5);
            resourceBO.setFileName(fileName);
            resourceBO.setCreateDate(new Date());
            resourceBO.setIsDelete(0);
            final Integer resourceId = fileDao.addResource(resourceBO);
            resourceBO.setResourceId(resourceId);
            redisDao.set(RedisPrefixConst.BREAKPOINT_PREFIX + fileMd5, JSONObject.toJSONString(resourceBO), RedisPrefixConst.EXPIRE);
        }

        //如果redis中不存在,但數據庫中存在,則存入redis中
        String breakpoint = redisDao.get(RedisPrefixConst.BREAKPOINT_PREFIX + fileMd5);
        if (StringUtils.isEmpty(breakpoint) && resourceBOList.size() > 0) {
            resourceBO = resourceBOList.get(0);
            redisDao.set(RedisPrefixConst.BREAKPOINT_PREFIX + fileMd5, JSONObject.toJSONString(resourceBO), RedisPrefixConst.EXPIRE);
        }

        // 若文件不存在則檢查文件所在目錄是否存在
        File fileFolder = new File(fileFolderPath);
        if (!fileFolder.exists()) {
            // 不存在創建該目錄 (目錄就是根據前端傳來的MD5值創建的)
            fileFolder.mkdirs();
        }
        return ResponseResult.success(null);
    }

    @Override
    public ResponseResult breakpointRenewal(MultipartFile file, String fileMd5, Integer chunk) {
        Map<String, String> ret = Maps.newHashMap();
        // 檢查分塊目錄是否存在
        String chunkFileFolderPath = this.getChunkFileFolderPath(fileMd5);
        File chunkFileFolder = new File(chunkFileFolderPath);
        if (!chunkFileFolder.exists()) {
            chunkFileFolder.mkdirs();
        }
        // 上傳文件輸入流
        File chunkFile = new File(chunkFileFolderPath + chunk);
        try (InputStream inputStream = file.getInputStream(); FileOutputStream outputStream = new FileOutputStream(chunkFile)) {
            IOUtils.copy(inputStream, outputStream);
            // redis中查找是否有fileMd5的分塊記錄(resId)
            String breakpoint = redisDao.get(RedisPrefixConst.BREAKPOINT_PREFIX + fileMd5);
            ResourceBO resourceBO = new ResourceBO();
            if (!StringUtils.isEmpty(breakpoint)) {
                // 存在分塊記錄說明資源正在上傳中,直接返回fileMd5對應的resId,且不再重複創建resource記錄
                resourceBO = JSONObject.parseObject(breakpoint, ResourceBO.class);
                ret.put("resId", String.valueOf(resourceBO.getResourceId()));
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return ResponseResult.success(ret);
    }

    @Override
    public ResponseResult checkChunk(String fileMd5, Integer chunk, Integer chunkSize) {
        // 檢查分塊文件是否存在
        String chunkFileFolderPath = this.getChunkFileFolderPath(fileMd5);
        // 分塊所在路徑+分塊的索引可定位具體分塊
        File chunkFile = new File(chunkFileFolderPath + chunk);
        if (chunkFile.exists() && chunkFile.length() == chunkSize) {
            return ResponseResult.success(true);
        }
        return ResponseResult.success(false);
    }

    @Override
    public ResponseResult mergeChunks(String fileMd5, String fileName, Long fileSize, String mimetype, String fileExt, LoginUserInfo user) {
        FileClient fileClient = ClientFactory.createClientByType(bootstrapConfig.getFileClientType());
        String chunkFileFolderPath = this.getChunkFileFolderPath(fileMd5);
        File chunkFileFolder = new File(chunkFileFolderPath);
        File[] files = chunkFileFolder.listFiles();
        final String filePath = this.getFilePath(fileMd5, fileExt);
        File mergeFile = new File(filePath);
        List<File> fileList = Arrays.asList(files);

        // 1. 合併分塊
        mergeFile = this.mergeFile(fileList, mergeFile);
        if (mergeFile == null) {
            return ResponseResult.fail(ResponseEnum.RESPONSE_CODE_MERGE_FILE_ERROR, null);
        }
        // 2、校驗文件MD5是否與前端傳入一致
        boolean checkResult = this.checkFileMd5(mergeFile, fileMd5);
        if (!checkResult) {
            return ResponseResult.fail(ResponseEnum.RESPONSE_CODE_VERIFY_FILE_ERROR, null);
        }

        // 3、刪除該文件所有分塊
        FileUtil.deleteDir(chunkFileFolderPath);
        // 4、在redis中獲取文件分塊記錄
        String breakpoint = redisDao.get(RedisPrefixConst.BREAKPOINT_PREFIX + fileMd5);
        if (StringUtils.isEmpty(breakpoint)) {
            return ResponseResult.fail("文件分塊不存在");
        }
        ResourceBO resourceBO = JSONObject.parseObject(breakpoint, ResourceBO.class);
        // 5、刪除redis分塊記錄
        redisDao.del(RedisPrefixConst.BREAKPOINT_PREFIX + fileMd5);

        // 6、組裝返回結果
        ret.put("filePath", getFileRelativePath(fileMd5, fileExt));
        ret.put("resId", String.valueOf(resourceBO.getResourceId()));
        return ResponseResult.success(ret);
    }

    /**
     * 合併文件
     *
     * @param chunkFileList
     * @param mergeFile
     * @return java.io.File
     * @author zxzhang
     * @date 2020/1/11
     */
    private File mergeFile(List<File> chunkFileList, File mergeFile) {
        try {
            // 有刪 無創建
            if (mergeFile.exists()) {
                mergeFile.delete();
            } else {
                mergeFile.createNewFile();
            }
            // 排序
            Collections.sort(chunkFileList, (o1, o2) -> {
                if (Integer.parseInt(o1.getName()) > Integer.parseInt(o2.getName())) {
                    return 1;
                }
                return -1;
            });

            byte[] b = new byte[1024];
            RandomAccessFile writeFile = new RandomAccessFile(mergeFile, "rw");
            for (File chunkFile : chunkFileList) {
                RandomAccessFile readFile = new RandomAccessFile(chunkFile, "r");
                int len = -1;
                while ((len = readFile.read(b)) != -1) {
                    writeFile.write(b, 0, len);
                }
                readFile.close();
            }
            writeFile.close();
            return mergeFile;
        } catch (IOException e) {
            e.printStackTrace();
            return null;
        }
    }

    /**
     * 校驗文件MD5
     *
     * @param mergeFile
     * @param md5
     * @return boolean
     * @author zxzhang
     * @date 2020/1/11
     */
    private boolean checkFileMd5(File mergeFile, String md5) {
        try {
            // 得到文件MD5
            FileInputStream inputStream = new FileInputStream(mergeFile);
            String md5Hex = DigestUtils.md5Hex(inputStream);

            if (StringUtils.equalsIgnoreCase(md5, md5Hex)) {
                return true;
            }

        } catch (Exception e) {
            e.printStackTrace();
        }
        return false;
    }

    /**
     * 獲取文件後綴
     *
     * @param fileName
     * @return java.lang.String
     * @author zxzhang
     * @date 2019/12/10
     */
    public String getExt(String fileName) {
        return fileName.substring(fileName.lastIndexOf(".") + 1);
    }

    /**
     * 獲取文件所在目錄
     *
     * @param filePath
     * @return java.lang.String
     * @author zxzhang
     * @date 2019/12/10
     */
    public String getFileDir(String filePath) {
        return filePath.substring(0, filePath.lastIndexOf(BootstrapConst.PATH_SEPARATOR));
    }

    /**
     * 獲取文件名
     *
     * @param filePath
     * @return java.lang.String
     * @author zxzhang
     * @date 2019/12/10
     */
    public String getFileName(String filePath) {
        return filePath.substring(filePath.lastIndexOf(BootstrapConst.PATH_SEPARATOR) + 1, filePath.lastIndexOf("."));
    }
}

4、FileManagementHelper代碼

package com.openailab.oascloud.file.common.helper;

import com.openailab.oascloud.file.common.config.BootstrapConfig;
import com.openailab.oascloud.file.common.consts.BootstrapConst;
import com.openailab.oascloud.file.common.enums.DocumentEnum;
import com.openailab.oascloud.file.common.enums.ImageEnum;
import com.openailab.oascloud.file.common.enums.ResourceTypeEnum;
import com.openailab.oascloud.file.common.enums.VedioEnum;
import com.openailab.oascloud.file.util.FileUtil;
import org.apache.commons.lang.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import java.io.*;
import java.text.SimpleDateFormat;
import java.util.Arrays;
import java.util.Date;
import java.util.LinkedList;

/**
 * @description:
 * @author: zhangzhixiang
 * @createDate: 2019/12/11
 * @version: 1.0
 */
@Component
public class FileManagementHelper {

    private static final Logger LOG = LoggerFactory.getLogger(FileManagementHelper.class);
    @Autowired
    private BootstrapConfig bootstrapConfig;

    /**
     * 根據文件後綴判斷類型
     *
     * @param ext
     * @return java.lang.Integer
     * @author zxzhang
     * @date 2019/12/10
     */
    public Integer judgeDocumentType(String ext) {
        //視頻類
        if (VedioEnum.containKey(ext) != null) {
            return ResourceTypeEnum.VIDEO.getCode();
        }
        //圖片類
        if (ImageEnum.containKey(ext) != null) {
            return ResourceTypeEnum.IMAGE.getCode();
        }
        //文檔類
        if (DocumentEnum.containKey(ext) != null) {
            return ResourceTypeEnum.FILE.getCode();
        }
        //未知
        return ResourceTypeEnum.OTHER.getCode();
    }

    /**
     * 生成隨機文件名稱
     *
     * @param ext
     * @return java.lang.String
     * @author zxzhang
     * @date 2019/12/10
     */
    public static String createFileName(String ext) {
        SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyyMMddHHmmss");
        return simpleDateFormat.format(new Date()) + (int) (Math.random() * 900 + 100) + ext;
    }

    /**
     * 獲取文件後綴
     *
     * @param fileName
     * @return java.lang.String
     * @author zxzhang
     * @date 2019/12/10
     */
    public String getExt(String fileName) {
        return fileName.substring(fileName.lastIndexOf(".") + 1);
    }

    /**
     * 獲取文件所在目錄
     *
     * @param filePath
     * @return java.lang.String
     * @author zxzhang
     * @date 2019/12/10
     */
    public String getFileDir(String filePath) {
        return filePath.substring(0, filePath.lastIndexOf(BootstrapConst.PATH_SEPARATOR));
    }

    /**
     * 獲取文件名
     *
     * @param filePath
     * @return java.lang.String
     * @author zxzhang
     * @date 2019/12/10
     */
    public String getFileName(String filePath) {
        return filePath.substring(filePath.lastIndexOf(BootstrapConst.PATH_SEPARATOR) + 1, filePath.lastIndexOf("."));
    }
}

5、RedisDao代碼

package com.openailab.oascloud.file.dao;

import com.openailab.oascloud.common.consts.ServiceNameConst;
import com.openailab.oascloud.common.model.ResponseResult;
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;

/**
 * @Classname: com.openailab.oascloud.datacenter.api.IRedisApi
 * @Description: Redis API
 * @Author: zxzhang
 * @Date: 2019/7/1
 */
@FeignClient(ServiceNameConst.OPENAILAB_DATA_CENTER_SERVICE)
public interface RedisDao {
    /**
     * @api {POST} /redis/set 普通緩存放入並設置過期時間
     * @apiGroup Redis
     * @apiVersion 0.1.0
     * @apiParam {String} key 鍵
     * @apiParam {String} value 值
     * @apiParam {long} expire 過期時間
     */
    @PostMapping("/redis/set")
    ResponseResult set(@RequestParam("key") String key, @RequestParam("value") String value, @RequestParam("expire") long expire);

    /**
     * @api {POST} /redis/get 普通緩存獲取
     * @apiGroup Redis
     * @apiVersion 0.1.0
     * @apiParam {String} key 鍵
     * @apiSuccess {String} value 值
     */
    @PostMapping("/redis/get")
    String get(@RequestParam("key") String key);

    /**
     * @api {POST} /redis/del 普通緩存刪除
     * @apiGroup Redis
     * @apiVersion 0.1.0
     * @apiParam {String} key 鍵
     */
    @PostMapping("/redis/del")
    ResponseResult del(@RequestParam("key") String key);

    /**
     * @api {POST} /redis/hset 存入Hash值並設置過期時間
     * @apiGroup Redis
     * @apiVersion 0.1.0
     * @apiParam {String} key 鍵
     * @apiParam {String} item 項
     * @apiParam {String} value 值
     * @apiParam {long} expire 過期時間
     */
    @PostMapping("/redis/hset")
    ResponseResult hset(@RequestParam("key") String key, @RequestParam("item") String item, @RequestParam("value") String value, @RequestParam("expire") long expire);

    /**
     * @api {POST} /redis/hget 獲取Hash值
     * @apiGroup Redis
     * @apiVersion 0.1.0
     * @apiParam {String} key 鍵
     * @apiParam {String} item 項
     * @apiSuccess {String} value 值
     * @apiSuccessExample {json} 成功示例
     * {"name":"張三","age":30}
     */
    @PostMapping("/redis/hget")
    Object hget(@RequestParam("key") String key, @RequestParam("item") String item);

    /**
     * @api {POST} /redis/hdel 刪除Hash值SaasAppKeyDao
     * @apiGroup Redis
     * @apiVersion 0.1.0
     * @apiParam {String} key 鍵
     * @apiParam {String} item 項
     */
    @PostMapping("/redis/hdel")
    ResponseResult hdel(@RequestParam("key") String key, @RequestParam("item") String item);
}

6、BootstrapConfig代碼

package com.openailab.oascloud.file.common.config;

import lombok.Data;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;

/**
 * @Classname: com.openailab.oascloud.security.common.config.BootstrapConsts
 * @Description: 引導類
 * @Author: zxzhang
 * @Date: 2019/10/8
 */
@Data
@Configuration
public class BootstrapConfig {

    @Value("${file.client.type}")
    private String fileClientType;

    @Value("${file.root}")
    private String fileRoot;

    @Value("${file.biz.file.upload}")
    private String uploadDir;

    @Value("${file.biz.file.download}")
    private String downloadDir;

    @Value("${file.biz.file.backup}")
    private String backupDir;

    @Value("${file.biz.file.tmp}")
    private String tmpDir;

    @Value("${file.biz.file.breakpoint}")
    private String breakpointDir;
}

7、application.properties

eureka.instance.instance-id=${spring.application.name}:${server.port}
eureka.instance.prefer-ip-address=true
eureka.client.serviceUrl.defaultZone=http://127.0.0.1:32001/eureka/
server.port=32018
spring.application.name=openailab-file-management
#file
file.client.type = ceph
file.root = /usr/local/oas/file
file.biz.file.upload = /upload
file.biz.file.download = /download
file.biz.file.backup = /backup
file.biz.file.tmp = /tmp
file.biz.file.breakpoint = /breakpoint
#ribbon
ribbon.ReadTimeout=600000
ribbon.ConnectTimeout=600000
#base
info.description=文件管理服務
[email protected]

spring.servlet.multipart.enabled=true
spring.servlet.multipart.max-file-size=5120MB
spring.servlet.multipart.max-request-size=5120MB

8、表結構

字段名 註釋 類型 長度 是否必填 是否主鍵
id 主鍵ID,sequence(course_resource_id_seq) int 32
type 資源類型,1:視頻;2:文檔;3:圖片 int 2
fileName 文件名稱 varchar 100
fileSize 文件大小 int 64
filePath 文件路徑 varchar 200
status 0:無需轉碼 1:轉碼中 2:已轉碼 3:未上傳完成 4:已上傳完成 -1:轉碼失敗 int 2
createDate 創建時間 timestamp 0
createUser 創建用戶 varchar 50
isDelete 是否刪除:0未刪除,1已刪除 int 2
userId 用戶ID int 32
fileMd5 文件唯一標識(webupload文件md5唯一標識) varchar 100

文件斷點續傳到此就介紹完了,完整前後端代碼奉上,傳送門:

斷點續傳前端代碼下載(WebUploader)

斷點續傳後端代碼下載(Java)

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章