Spring Batch入門實例

在對Spring Batch深入學習前，先寫一個小的demo，增加對Spring Batch的瞭解

相關信息

開發環境：
JDK 1.8.0_201
apache-maven-3.3.9
IntelliJ IDEA 2019.2.1 (Community Edition)

版本信息
Spring Batch 3.0.10.RELEASE
Spring test 4.3.21.RELEASE

相關概念

領域對象	描述
Job Repository	作業倉庫，負責Job,Step執行過程中的狀態保存
Job launcher	作業調度器，提供執行Job的入口
Job	作業，由多個Step組成，封裝整個批處理操作
Step	作業步，Job的一個執行環節，由多個或者一個Step組裝成Job
Tasklet	Step中具體執行邏輯的操作，可以重複執行，可以設置具體的同步、異步操作等
Chunk	給定數量Item的集合，可以定義對Chunk的讀操作、處理操作、寫操作，提交間隔等，這是Spring Batch框架的一個重要特徵
Item	一條數據記錄
ItemReader	從數據源（文件系統、數據庫、隊列等）中讀取Item
ItemProcessor	在Item寫入數據源之前，對數據進行處理（如：數據清洗、數據轉換、數據過濾、數據校驗等）
ItemWriter	將Item批量寫入數據源（文件系統、數據庫、隊列等）

demo背景

通過Spring Batch來讀取csv文件，讀完後再將讀到的數據寫入另一個csv文件

代碼展示

項目結構

配置pom.xml文件

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.et</groupId>
    <artifactId>spring-batch-demo</artifactId>
    <version>1.0-SNAPSHOT</version>

    <name>spring-batch-demo</name>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.batch</groupId>
            <artifactId>spring-batch-core</artifactId>
            <version>3.0.10.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>4.3.21.RELEASE</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <pluginManagement><!-- lock down plugins versions to avoid using Maven defaults (may be moved to parent pom) -->
            <plugins>
                <!-- clean lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#clean_Lifecycle -->
                <plugin>
                    <artifactId>maven-clean-plugin</artifactId>
                    <version>3.1.0</version>
                </plugin>
                <!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->
                <plugin>
                    <artifactId>maven-resources-plugin</artifactId>
                    <version>3.0.2</version>
                </plugin>
                <plugin>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.8.0</version>
                </plugin>
                <plugin>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.22.1</version>
                </plugin>
                <plugin>
                    <artifactId>maven-jar-plugin</artifactId>
                    <version>3.0.2</version>
                </plugin>
                <plugin>
                    <artifactId>maven-install-plugin</artifactId>
                    <version>2.5.2</version>
                </plugin>
                <plugin>
                    <artifactId>maven-deploy-plugin</artifactId>
                    <version>2.8.2</version>
                </plugin>
                <!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->
                <plugin>
                    <artifactId>maven-site-plugin</artifactId>
                    <version>3.7.1</version>
                </plugin>
                <plugin>
                    <artifactId>maven-project-info-reports-plugin</artifactId>
                    <version>3.0.0</version>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</project>

因爲Spring batch自己的pom中有依賴很多Spring核心包，所以在pom中沒有配置。

定義領域對象

根據要需求定義相關對象

public class StudentInfo {
    private String name;
    private int age;
    private String gender;
    private String birthday;
    private String address;

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public String getGender() {
        return gender;
    }

    public void setGender(String gender) {
        this.gender = gender;
    }

    public String getBirthday() {
        return birthday;
    }

    public void setBirthday(String birthday) {
        this.birthday = birthday;
    }

    public String getAddress() {
        return address;
    }

    public void setAddress(String address) {
        this.address = address;
    }
}

配置job

在Spring配置文件中定義批處理任務。
需要在xml文件頭添加spring batch相關申明

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:context="http://www.springframework.org/schema/context"
       xmlns:batch="http://www.springframework.org/schema/batch"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                    http://www.springframework.org/schema/beans/spring-beans.xsd
                    http://www.springframework.org/schema/context
                    http://www.springframework.org/schema/context/spring-context.xsd
                    http://www.springframework.org/schema/batch
                    http://www.springframework.org/schema/batch/spring-batch-3.0.xsd">

定義job基礎配置

<!--採用了內存方式記錄job執行期產生的狀態信息-->
    <bean id="jobRepository"
          class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"></bean>

    <!--作業調度器，用來啓動Job-->
    <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository"/>
    </bean>

    <bean id="transactionManager"
          class="org.springframework.batch.support.transaction.ResourcelessTransactionManager"></bean>

定義job

<batch:job id="studentJob">
        <batch:step id="studentStep">
            <batch:tasklet transaction-manager="transactionManager">
                <!--commit-interval=2 表示提交間隔的大小，即每處理2條數據，進行一次寫操作-->
                <batch:chunk reader="csvItemReader" writer="csvItemWriter" processor="studentProcessor"
                             commit-interval="2"></batch:chunk>
            </batch:tasklet>
        </batch:step>
    </batch:job>

配置ItemReader

<bean id="csvItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
        <property name="resource" value="classpath:basic/test.csv"/>
        <property name="lineMapper">
            <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
                <property name="lineTokenizer" ref="lineTokenizer"/>
                <property name="fieldSetMapper">
                    <bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper">
                        <property name="prototypeBeanName" value="studentInfo"/>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>

    <bean id="lineTokenizer" class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
        <property name="delimiter" value=","/>
        <property name="names">
            <list>
                <value>name</value>
                <value>age</value>
                <value>gender</value>
                <value>birthday</value>
                <value>address</value>
            </list>
        </property>
    </bean>

配置ItemProcessor

<bean id="studentProcessor" scope="step" class="com.et.basic.StudentInfoProcessor"></bean>

新建java類，實現process功能

import org.springframework.batch.item.ItemProcessor;

public class StudentInfoProcessor implements ItemProcessor<StudentInfo, StudentInfo> {
    @Override
    public StudentInfo process(StudentInfo studentInfo) throws Exception {
        System.out.println(studentInfo.toString());
        return studentInfo;
    }
}

配置ItemWriter

<bean id="csvItemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter" scope="step">
        <property name="resource" value="file:target/basic/outputFile.csv"/>
        <property name="lineAggregator">
            <bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
                <property name="delimiter" value=","/>
                <property name="fieldExtractor">
                    <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
                        <property name="names" value="name,age,gender,birthday,address"/>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>

執行job

準備好測試文件，就可以通過下面的方式進行測試，同時需要觀察console框中的輸出信息。

java調用

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class JobLaunch {
    public static void main(String[] args) {
        ApplicationContext context = new ClassPathXmlApplicationContext("basic/job.xml");
        JobLauncher launcher = (JobLauncher) context.getBean("jobLauncher");
        Job job = (Job) context.getBean("studentJob");

        try {
            JobExecution result = launcher.run(job, new JobParameters());
            System.out.println(result.toString());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

JUnit單元測試

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = ("/basic/job.xml"))
public class JobLaunchTest {
    @Autowired
    private JobLauncher jobLauncher;
    @Autowired
    @Qualifier("studentJob")
    private Job job;

    @Before
    public void setUp() throws Exception {
    }

    @After
    public void tearDown() throws Exception {
    }

    @Test
    public void batchTest() throws Exception {
        JobExecution result = jobLauncher.run(job, new JobParameters());
        System.out.println(result.toString());
    }
}

總結

從代碼上看，整個過程，我認爲還是挺清楚地，所以沒有做太多的解釋。
參考鏈接：

Spring Batch 3.0.x Introduction

完整的案例，已經上傳至github。