我要傳的是一個每天9-11點生成的500M的文件夾,WebService服務是以接口形式傳輸數據,無法直接傳遞文件,我將其壓縮爲zip格式,然後轉換爲字符串,以對象形式傳遞,在客戶端解析解壓整合成原來的文件,從而實現文件夾的傳輸。
文件結構如下
1.準備工作
pom
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.3.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.gwhn</groupId>
<artifactId>webservice</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>webservice</name>
<packaging>war</packaging>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web-services</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<!-- 移除嵌入式tomcat插件 -->
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>javax.servlet-api</artifactId>
<version>4.0.1</version>
<scope>provided</scope></dependency>
<!--WerbService CXF依賴-->
<!-- https://mvnrepository.com/artifact/org.apache.cxf/cxf-rt-frontend-jaxws -->
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-frontend-jaxws</artifactId>
<version>3.2.6</version>
</dependency>
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http</artifactId>
<version>3.2.6</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.5</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.junit.jupiter/junit-jupiter-api -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.1.0-M2</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.axis2.wso2/axis2 -->
<!-- https://mvnrepository.com/artifact/org.apache.axis2/axis2 -->
<dependency>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2</artifactId>
<version>1.7.9</version>
<type>pom</type>
</dependency>
<!-- https://mvnrepository.com/artifact/net.iharder/base64 -->
<dependency>
<groupId>net.iharder</groupId>
<artifactId>base64</artifactId>
<version>2.3.9</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
//建立對象
public class MicapsData implements Serializable{
private static final long serialVersionUID = -2506579878568411053L;
private String fileName;//micaps文件名
private String encodedFileString;//每次傳輸的字符串
}
//日期工具
public class DateUtil {
/**
* 獲取當前時間數字
*/
public String getDate() {
Date date = new Date();
SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyyMMdd HH:mm:ss");//20190822 12:50:02
String day = simpleDateFormat.format(date);
return day.toString();
}
/**
* 獲取前推,後推日期日期yyyyMMdd
* */
public String getDate(int num){
Date date = new Date();//獲取當前時間
Calendar calendar = new GregorianCalendar();
calendar.setTime(date);
calendar.add(Calendar.DATE,num);//日期往前推一天
date = calendar.getTime();
SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyMMdd");
return simpleDateFormat.format(date).toString();
}
}
//路徑配置文件
pathconfig.txt
每日micaps文件夾路徑:D:\\zhpfile\\ideaworkspace
臨時文件夾路徑:D:\\zhpfile\\ideaworkspace\\gz
日誌存放路徑:D:\\zhpfile\\ideaworkspace\\gwhnWebServiceLog
其他新增配置請聯繫xxxx
路徑配置文件在resources文件夾下,我寫了一個類專門讀取這些路徑
public class ReadConfig {
/**
* 讀取配置文件所有字符
* */
public String readConfig() throws IOException,Exception{
BufferedReader bufferedReader = null;
URL url = ReadConfig.class.getClassLoader().getResource("pathconfig.txt");//文件路徑
File file = new File(url.getFile());
FileReader fileReader = new FileReader(url.getFile());//得到配置文件
bufferedReader = new BufferedReader(new InputStreamReader(new FileInputStream(file.getPath()), "utf-8")); ;//用字符流讀取
String word = "";
String str="";
while((str=bufferedReader.readLine())!=null) { //●判斷最後一行不存在,爲空
word = word + str;
}
bufferedReader.close();//關閉字符流
fileReader.close();
return word;
}
/**
* 從pathconfig中提取每日micaps文件夾上一級路徑
* */
public String getMicapPath() throws Exception{
String configPath = readConfig().substring(readConfig().indexOf("每日micaps文件夾路徑:")+14,readConfig().indexOf("臨時文件夾路徑:"));
DateUtil dateUtil = new DateUtil();
return configPath+"\\"+dateUtil.getDate(-1);//業務化運行使用的路徑
// return "D:\\zhpfile\\ideaworkspace\\19102820";
}
/**
* 從pathconfig中提取每日臨時文件夾路徑
* */
public String getTempPath() throws Exception{
return readConfig().substring(readConfig().indexOf("臨時文件夾路徑:")+8,readConfig().indexOf("日誌存放路徑:"));//業務化運行使用的路徑
// return "D:\\zhpfile\\ideaworkspace\\gz";
}
public String getLogPath() throws Exception{
return readConfig().substring(readConfig().indexOf("日誌存放路徑:")+7,readConfig().indexOf("其他新增配置"));//業務化運行使用的路徑
}
}
2.文件處理
public interface ZipFileService {
public void FileToZip(String srcDir, OutputStream out, boolean KeepDirStructure) throws RuntimeException;
public void ZipToFile(File srcFile, String destDirPath);
public void compress(File sourceFile, ZipOutputStream zos, String name,
boolean KeepDirStructure) throws Exception;
public void copyFile(File fromFile,File toFile) throws Exception;
public String[] getFileList(File file) ;
public void deleteDirectory(File file);
}
public class ZipFileServiceImpl implements ZipFileService {
private static final int BUFFER_SIZE = 2 * 1024;
/**
* 原始壓縮成ZIP 方法
* 由於文件太大無法傳輸,所以我修改成先壓爲爲幾個包,而後分開傳輸,到客戶端解壓到同一文件夾下
* @param srcDir 壓縮文件夾路徑
* @param out 壓縮文件輸出流
* @param KeepDirStructure 是否保留原來的目錄結構,true:保留目錄結構;
* false:所有文件跑到壓縮包根目錄下(注意:不保留目錄結構可能會出現同名文件,會壓縮失敗)
* @throws RuntimeException 壓縮失敗會拋出運行時異常
*/
public void FileToZip(String srcDir, OutputStream out, boolean KeepDirStructure) throws RuntimeException {
long start = System.currentTimeMillis();
ZipOutputStream zos = null;
try {
zos = new ZipOutputStream(out);
File sourceFile = new File(srcDir);
compress(sourceFile, zos, sourceFile.getName(), KeepDirStructure);
long end = System.currentTimeMillis();
System.out.println(srcDir+"壓縮完成,耗時:" + (end - start) + " ms");
} catch (Exception e) {
throw new RuntimeException("zip error from ZipUtils", e);
} finally {
if (zos != null) {
try {
zos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
/**
* zip解壓
* @param srcFile zip源文件
* @param destDirPath 解壓後的目標文件夾
* @throws RuntimeException 解壓失敗會拋出運行時異常
*/
@Override
public void ZipToFile(File srcFile, String destDirPath) {
long start = System.currentTimeMillis();
if (!srcFile.exists()) { // 判斷源文件是否存在
throw new RuntimeException(srcFile.getPath() + "所指文件不存在");
}
// 開始解壓
ZipFile zipFile = null;
try {
zipFile = new ZipFile(srcFile);
Enumeration<?> entries = zipFile.entries();
while (entries.hasMoreElements()) {
ZipEntry entry = (ZipEntry) entries.nextElement();
System.out.println("解壓" + entry.getName());
if (entry.isDirectory()) { // 如果是文件夾,就創建個文件夾
String dirPath = destDirPath + "/" + entry.getName();
File dir = new File(dirPath);
dir.mkdirs();
} else {
File targetFile = new File(destDirPath + "/" + entry.getName());// 如果是文件,就先創建一個文件,然後用io流把內容copy過去
if(!targetFile.getParentFile().exists()){ // 保證這個文件的父文件夾必須要存在
targetFile.getParentFile().mkdirs();
}
targetFile.createNewFile();
// 將壓縮文件內容寫入到這個文件中
InputStream is = zipFile.getInputStream(entry);
FileOutputStream fos = new FileOutputStream(targetFile);
int len;
byte[] buf = new byte[BUFFER_SIZE];
while ((len = is.read(buf)) != -1) {
fos.write(buf, 0, len);
}
fos.close();// 關流順序,先打開的後關閉
is.close();
}
}
long end = System.currentTimeMillis();
System.out.println("解壓完成,耗時:" + (end - start) +" ms");
} catch (Exception e) {
throw new RuntimeException("unzip error from ZipUtils", e);
} finally {
if(zipFile != null){
try {
zipFile.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
/**
* 遞歸壓縮方法
* @param sourceFile 源文件
* @param zos zip輸出流
* @param name 壓縮後的名稱
* @param KeepDirStructure 是否保留原來的目錄結構,true:保留目錄結構;
* false:所有文件跑到壓縮包根目錄下(注意:不保留目錄結構可能會出現同名文件,會壓縮失敗)
* @throws Exception
*/
public void compress(File sourceFile, ZipOutputStream zos, String name,
boolean KeepDirStructure) throws Exception {
byte[] buf = new byte[BUFFER_SIZE];
if (sourceFile.isFile()) {
// 向zip輸出流中添加一個zip實體,構造器中name爲zip實體的文件的名字
zos.putNextEntry(new ZipEntry(name));
// copy文件到zip輸出流中
int len;
FileInputStream in = new FileInputStream(sourceFile);
while ((len = in.read(buf)) != -1) {
zos.write(buf, 0, len);
}
// Complete the entry
zos.closeEntry();
in.close();
} else {
File[] listFiles = sourceFile.listFiles();
if (listFiles == null || listFiles.length == 0) {
// 需要保留原來的文件結構時,需要對空文件夾進行處理
if (KeepDirStructure) {
// 空文件夾的處理
zos.putNextEntry(new ZipEntry(name + "/"));
// 沒有文件,不需要文件的copy
zos.closeEntry();
}
} else {
for (File file : listFiles) {
// 判斷是否需要保留原來的文件結構
if (KeepDirStructure) {
// 注意:file.getName()前面需要帶上父文件夾的名字加一斜槓,
// 不然最後壓縮包中就不能保留原來的文件結構,即:所有文件都跑到壓縮包根目錄下了
compress(file, zos, name + "/" + file.getName(), KeepDirStructure);
} else {
compress(file, zos, file.getName(), KeepDirStructure);
}
}
}
}
}
/**
* 複製文件
* @param fromFile
* @param toFile
* <br/>
* 2016年12月19日 下午3:31:50
* @throws IOException
*/
@Override
public void copyFile(File fromFile, File toFile) throws Exception {
FileInputStream ins = new FileInputStream(fromFile);
FileOutputStream out = new FileOutputStream(toFile);
byte[] b = new byte[1024];
int n=0;
while((n=ins.read(b))!=-1){
out.write(b, 0, n);
}
ins.close();
out.close();
}
/**
* 獲取一個文件夾下所有文件名
* */
@Override
public String[] getFileList(File file) {
return new String[0];
}
// 刪除文件夾
@Override
public void deleteDirectory(File file) {
if (file.isFile()) {// 表示該文件不是文件夾
file.delete();
} else {
String[] childFilePaths = file.list();// 首先得到當前的路徑
for (String childFilePath : childFilePaths) {
File childFile = new File(file.getAbsolutePath() + "/" + childFilePath);
deleteDirectory(childFile);
}
file.delete();
}
}
}
3.要暴露服務的接口,重點來了
注意文件轉字符串的類在net.iharder下,
接口路徑如下
@WebService(targetNamespace = “http://service.gwhn.com/”, endpointInterface = “com.gwhn.service.FileService”)
前面是服務空間,倒序到服務爲止,後面是正序接口類,寫到要使用的接口
@WebService
public interface FileService {
@WebMethod
public MicapsData getZipMicapsdata(String name,String date) throws Exception;
@WebMethod
public int deleteTempFiles(int sendOver)throws Exception;
}
import com.gwhn.dao.FileDao;
import com.gwhn.dao.impl.FileDaoImpl;
import com.gwhn.entity.MicapsData;
import com.gwhn.service.FileService;
import com.gwhn.service.ZipFileService;
import com.gwhn.util.DateUtil;
import com.gwhn.util.LogUtil;
import com.gwhn.util.ReadConfig;
import net.iharder.*;
import net.iharder.Base64;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import javax.annotation.Resource;
import javax.jws.WebService;
import java.io.*;
import java.util.*;
/**
* Created by DLuser on 2019/10/30.
*/
@WebService(targetNamespace = "http://service.gwhn.com/", endpointInterface = "com.gwhn.service.FileService")
@Component
public class FileServiceImpl implements FileService {
ReadConfig readConfig = new ReadConfig();
private Map<String, MicapsData> micapsDataMap = new HashMap<String, MicapsData>();
String filename = "";
public FileServiceImpl() throws Exception {
}
/**
* 根據文件名獲取當天預測文件形成的字符串
*/
@Override
public MicapsData getZipMicapsdata(String name, String date) throws Exception {
ZipFileService zipFileService = new ZipFileServiceImpl();
DateUtil dateUtil = new DateUtil();
try{
if (name.equals("1") && date.equals("1")) {
zipFileService.deleteDirectory(new File(readConfig.getTempPath()));
LogUtil logUtil = new LogUtil();
System.out.println("本次傳輸完畢");
logUtil.writeLog(dateUtil.getDate()+name+"傳輸成功");
return null;//傳來傳輸完畢信號,處理緩存
} else {
String tempPath = readConfig.getTempPath();//獲取臨時文件夾路徑D:\zhpfile\ideaworkspace\gz
System.out.println("臨時文件夾路徑:" + tempPath);
String micapsPath = readConfig.getMicapPath();//獲取micaps文件夾路徑D:\zhpfile\ideaworkspace\19102820
System.out.println("micaps文件夾路徑:" + micapsPath);
String dateString = dateUtil.getDate(-1);//19082220
File file = new File(tempPath);
if (!file.exists()) {
file.mkdirs();
}
// File file1 = new File(tempPath + "\\" + date + "log.txt");
System.out.println("對方傳遞的當天日期字符串:" + date);
if (date.equals(dateString)) {
file.mkdirs();
if (name.equals(dateString + "10FG6.zip")) {
System.out.println("開始複製"+micapsPath+"中零散數據到"+tempPath);
copyFiles(micapsPath, tempPath, dateString);//複製四中數據到指定目錄下
zipFiles(micapsPath, tempPath, dateString); /**分別壓縮四個文件*/
}
/**將當天的各項micaps數據存入映射*/
List<String> filesList = getFilNameStringList();
for (int i = 0; i < filesList.size(); i++) {
file = new File(tempPath + "\\" + dateString + filesList.get(i) + ".zip");
MicapsData micapsData = new MicapsData();
micapsData.setFileName(file.getName());
FileDao fileDao = new FileDaoImpl();
String fileString = fileDao.getFileByteString(file);
micapsData.setEncodedFileString(fileDao.getFileByteString(file));
micapsDataMap.put(file.getName(), micapsData);
}
}
filename = name;
System.out.println("對方請求的的文件名:" + filename);
MicapsData micapsData = micapsDataMap.get(filename);
return micapsData;
}
}catch (Exception e){
DateUtil dateUtil1 = new DateUtil();
LogUtil logUtil = new LogUtil();
logUtil.writeLog(dateUtil.getDate()+"傳輸失敗");
System.out.println("傳輸失敗");
}
return null;
}
/**
* 接收到對方傳來的傳輸完畢信息,刪除緩存文件夾
*/
@Override
public int deleteTempFiles(int sendOver) throws Exception {
if (sendOver == 1) {
ZipFileService zipFileService = new ZipFileServiceImpl();
File file = new File(readConfig.getTempPath());
zipFileService.deleteDirectory(file);
}
return 1;
}
/**
* 複製當天對應的文件
*/
public void copyFiles(String micapsPath, String tempPath, String dateString) throws Exception {
ZipFileService zipFileService = new ZipFileServiceImpl();
File file = new File(micapsPath);
List<String> micapsFileList = new ArrayList<String>();
File[] fileList = file.listFiles();
for (int i = 0; i < fileList.length; i++) {
if (!fileList[i].isDirectory()) {
File srcfile = new File(fileList[i].getPath());
// File gzFile = new File("D:\\zhpfile\\ideaworkspace\\gz19102820\\");
File gzFile = new File(tempPath + "\\" + dateString + "gz");
if (!gzFile.exists()) {
gzFile.mkdirs();
}
File tofile = new File(gzFile.getPath() + "\\" + srcfile.getName());
zipFileService.copyFile(srcfile, tofile);
}
}
FileOutputStream out = new FileOutputStream(new File(tempPath + "\\" + dateString + "gz.zip"));
zipFileService.FileToZip(tempPath + "\\" + dateString + "gz", out, true);
}
/**
* 生成文件夾字符集合
*/
public List<String> getFilNameStringList() {
List<String> pathList = new ArrayList<String>();
pathList.add("10FG6");
pathList.add("WIND_10M");
pathList.add("WIND10M");
pathList.add("gz");
return pathList;
}
/**
* 壓縮四個文件夾到指定目錄下
*/
public void zipFiles(String micapsPath, String tempPath, String dateString) throws Exception {
ZipFileService zipFileService = new ZipFileServiceImpl();
OutputStream outputStream = new FileOutputStream(tempPath + "\\" + dateString + "10FG6.zip");
zipFileService.FileToZip(micapsPath + "\\10FG6", outputStream, true);
outputStream = new FileOutputStream(tempPath + "\\" + dateString + "WIND_10M.zip");
zipFileService.FileToZip(micapsPath + "\\WIND_10M", outputStream, true);
outputStream = new FileOutputStream(tempPath + "\\" + dateString + "WIND10M.zip");
zipFileService.FileToZip(micapsPath + "\\WIND10M", outputStream, true);
}
}
將文件轉換爲字符串!!!
@Component
public class FileDaoImpl implements FileDao {
/**
* 將文件轉換爲字符串
*/
@Override
public String getFileByteString(File file) throws Exception {
String encodedFileString = "";
InputStream inputStream = new FileInputStream(file);
long length = file.length();// 取得文件大小
byte[] bytes = new byte[(int) length];// 根據大小創建字節數組,每次大約20M
int offset = 0;// 讀取文件內容到字節數組
int numRead = 0;
while (offset < bytes.length
&& (numRead = inputStream.read(bytes, offset, bytes.length - offset)) >= 0) {
offset += numRead;
}
if (offset < bytes.length) {// 讀取完畢的校驗
throw new IOException("不能完全讀取文件:" + file.getName());
}
inputStream.close();
encodedFileString = Base64.encodeBytes(bytes);
return encodedFileString;
}
}
5,webservice服務配置,這是關鍵
import com.gwhn.service.FileService;
import com.gwhn.service.impl.FileServiceImpl;
import org.apache.cxf.Bus;
import org.apache.cxf.bus.spring.SpringBus;
import org.apache.cxf.jaxws.EndpointImpl;
import org.apache.cxf.transport.servlet.CXFServlet;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import javax.xml.ws.Endpoint;
/**
* Created by DLuser on 2019/10/28.
*/
@Configuration
public class WebServiceConfig {
@Bean
public ServletRegistrationBean dispatcherServlet() {
return new ServletRegistrationBean(new CXFServlet(), "/service/*");//發佈服務名稱
}
@Bean(name = Bus.DEFAULT_BUS_ID)
public SpringBus springBus() {
return new SpringBus();
}
@Bean
public FileService fileService() throws Exception{
return new FileServiceImpl();
}
@Bean
public Endpoint endpoint() throws Exception{
EndpointImpl endpoint = new EndpointImpl(springBus(), fileService());//綁定要發佈的服務
endpoint.publish("/micapsdata"); //顯示要發佈的名稱
return endpoint;
}
}
然後是日誌書寫,這個需求是有時間和是否成功,我寫的很簡單
public class LogUtil {
/**
* 獲取日誌文件
*/
public File getLogFile() throws Exception {
ReadConfig readConfig = new ReadConfig();
File file = new File(readConfig.getLogPath()+".txt");
if (!file.exists()) {
file.createNewFile();
}
return file;
}
/**
* 獲取日誌文件所有字符,本方法建議不要和readConfig中的合併
*/
public String getLogString() throws Exception {
File file = getLogFile();
BufferedReader bufferedReader = null;
FileReader fileReader = new FileReader(file);//得到配置文件
bufferedReader = new BufferedReader(new InputStreamReader(new FileInputStream(file.getPath()), "utf-8"));
;//用字符流讀取
String word = "";
String str = "";
while ((str = bufferedReader.readLine()) != null) { //●判斷最後一行不存在,爲空
word = word + str;
}
bufferedReader.close();//關閉字符流
fileReader.close();
return word;
}
/**
* 獲取日誌文件'所有字符
*/
public void writeLog(String content) throws Exception {
FileOutputStream fileOutputStream = null;
ReadConfig readConfig = new ReadConfig();
File file = getLogFile();
String oldContent = getLogString();
fileOutputStream = new FileOutputStream(file);
fileOutputStream.write((oldContent+"\t\t"+content).getBytes());
fileOutputStream.flush();
fileOutputStream.close();
}
}
最後是啓動類,是已經可以打包扔到tomcat的那種
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.servlet.support.SpringBootServletInitializer;
@SpringBootApplication
public class WebserviceApplication extends SpringBootServletInitializer {
public static void main(String[] args) {
SpringApplication.run(WebserviceApplication.class, args);
}
@Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
// 這裏sources的類就是啓動類
return builder.sources(WebserviceApplication.class);
}
}
接下來寫客戶端
需求是定時檢查固定目錄是否有當天文件,沒有的話發送請求到服務端請求文件