spring4開發SpringBatch 樣例 -配置文件版

原創 peipei巴比 2019-04-04 02:46

本文根據《SpringBatch批處理框架》一書作者劉相的源碼升級spring4.0，源碼爲3.2版本
經測試可用
提示：本文源碼比較多
遇到的問題總結在後面

配置

數據庫初始化 create-tables-mysql.sql

DROP TABLE IF EXISTS t_credit;
DROP TABLE IF EXISTS t_destcredit;

CREATE TABLE t_credit
    (ID VARCHAR(64),
        ACCOUNTID VARCHAR(20),
        NAME VARCHAR(10),
        AMOUNT NUMERIC(10,2),
        DATE VARCHAR(20),
        ADDRESS VARCHAR(128),
        FLAG VARCHAR(10),
        primary key (ID)
    )
    ENGINE=InnoDB DEFAULT CHARSET=utf8;
    
CREATE TABLE t_destcredit
    (ID VARCHAR(64),
        ACCOUNTID VARCHAR(20),
        NAME VARCHAR(10),
        AMOUNT NUMERIC(10,2),
        DATE VARCHAR(20),
        ADDRESS VARCHAR(128),
        primary key (ID)
    )
    ENGINE=InnoDB DEFAULT CHARSET=utf8;
    
    
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('01','4047390012345678','tom',100.00,'2013-2-2 12:00:08','Lu Jia Zui road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('02','4047390012345678','tom',320.00,'2013-2-3 10:35:21','Lu Jia Zui road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('03','4047390012345678','tom',674.70,'2013-2-6 16:26:49','South Linyi road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('04','4047390012345678','tom',793.20,'2013-2-9 15:15:37','Longyang road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('05','4047390012345678','tom',360.00,'2013-2-11 11:12:38','Longyang road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('06','4047390012345678','tom',100.00,'2013-2-2 12:00:08','Lu Jia Zui road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('07','4047390012345678','tom',320.00,'2013-2-3 10:35:21','Lu Jia Zui road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('08','4047390012345678','tom',674.70,'2013-2-6 16:26:49','South Linyi road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('09','4047390012345678','tom',793.20,'2013-2-9 15:15:37','Longyang road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('10','4047390012345678','tom',360.00,'2013-2-11 11:12:38','Longyang road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('11','4047390012345678','tom',100.00,'2013-2-2 12:00:08','Lu Jia Zui road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('12','4047390012345678','tom',320.00,'2013-2-3 10:35:21','Lu Jia Zui road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('13','4047390012345678','tom',674.70,'2013-2-6 16:26:49','South Linyi road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('14','4047390012345678','tom',793.20,'2013-2-9 15:15:37','Longyang road','false');
INSERT INTO t_credit(ID,ACCOUNTID,NAME,AMOUNT,DATE,ADDRESS,FLAG) VALUES('15','4047390012345678','tom',360.00,'2013-2-11 11:12:38','Longyang road','false');

數據庫連接配置 batch-mysql.properties

datasource.driver=com.mysql.jdbc.Driver
datasource.url=jdbc:mysql://127.0.0.1:3306/test1289
datasource.username=root
datasource.password=000000

Spring DB驅動配置 job-context.xml

<?xml version="1.0" encoding="UTF-8"?>
<bean:beans xmlns="http://www.springframework.org/schema/batch"    
    xmlns:bean="http://www.springframework.org/schema/beans" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    
    xmlns:p="http://www.springframework.org/schema/p" 
    xmlns:tx="http://www.springframework.org/schema/tx"    
    xmlns:aop="http://www.springframework.org/schema/aop"
    xmlns:task="http://www.springframework.org/schema/task" 
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:jdbc="http://www.springframework.org/schema/jdbc"    
    xsi:schemaLocation="http://www.springframework.org/schema/beans 
    http://www.springframework.org/schema/beans/spring-beans-4.0.xsd  
    http://www.springframework.org/schema/tx 
    http://www.springframework.org/schema/tx/spring-tx-4.0.xsd  
    http://www.springframework.org/schema/aop 
    http://www.springframework.org/schema/aop/spring-aop-4.0.xsd  
    http://www.springframework.org/schema/context 
    http://www.springframework.org/schema/context/spring-context-3.0.xsd
    http://www.springframework.org/schema/batch 
    http://www.springframework.org/schema/batch/spring-batch-3.0.xsd
    http://www.springframework.org/schema/task 
    http://www.springframework.org/schema/task/spring-task-4.0.xsd
    http://www.springframework.org/schema/jdbc 
    http://www.springframework.org/schema/jdbc/spring-jdbc-4.0.xsd">
    <!-- 
    <jdbc:initialize-database data-source="dataSource">
    	<jdbc:script location="classpath:db/create-tables-mysql.sql" />
  	</jdbc:initialize-database>
  	 -->
    <!-- 
    <bean:bean id="simpleJdbcTemplate" class="org.springframework.jdbc.core.simple.SimpleJdbcTemplate">
    	<bean:constructor-arg ref="dataSource" />
    </bean:bean>
-->
  	<bean:bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
    	<bean:constructor-arg ref="dataSource" />
  	</bean:bean>
</bean:beans>

Bean定義 job-context.xml

<?xml version="1.0" encoding="UTF-8"?>
<bean:beans xmlns="http://www.springframework.org/schema/batch"    
    xmlns:bean="http://www.springframework.org/schema/beans" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    
    xmlns:p="http://www.springframework.org/schema/p" 
    xmlns:tx="http://www.springframework.org/schema/tx"    
    xmlns:aop="http://www.springframework.org/schema/aop"
    xmlns:task="http://www.springframework.org/schema/task" 
    xmlns:context="http://www.springframework.org/schema/context"    
    xsi:schemaLocation="http://www.springframework.org/schema/beans 
    http://www.springframework.org/schema/beans/spring-beans-4.0.xsd  
    http://www.springframework.org/schema/tx 
    http://www.springframework.org/schema/tx/spring-tx-4.0.xsd  
    http://www.springframework.org/schema/aop 
    http://www.springframework.org/schema/aop/spring-aop-4.0.xsd  
    http://www.springframework.org/schema/context 
    http://www.springframework.org/schema/context/spring-context-3.0.xsd
    http://www.springframework.org/schema/batch 
    http://www.springframework.org/schema/batch/spring-batch-3.0.xsd
    http://www.springframework.org/schema/task 
    http://www.springframework.org/schema/task/spring-task-4.0.xsd">
    
    <context:property-placeholder location="classpath:db/batch-mysql.properties" />
   
    <bean:bean 
	    class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
		<bean:property name="jobRegistry" ref="jobRegistry" />
	</bean:bean>
	
	<bean:bean id="stepLocator" 
		class="org.springframework.batch.integration.partition.BeanFactoryStepLocator" />
	
    <bean:bean id="jobRegistry" 
        class="org.springframework.batch.core.configuration.support.MapJobRegistry" />
    
    <bean:bean id="jobExplorer"
		class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean">
		<bean:property name="dataSource" ref="dataSource" />	
	</bean:bean>
	
    <bean:bean id="jobOperator" 
        class="org.springframework.batch.core.launch.support.SimpleJobOperator">
    	<bean:property name="jobRepository" ref="jobRepository" />
    	<bean:property name="jobLauncher" ref="jobLauncher" />    	
    	<bean:property name="jobRegistry" ref="jobRegistry" />
    	<bean:property name="jobExplorer" ref="jobExplorer" />
    </bean:bean>

<!-- "SERIALIZABLE" -->	
	<job-repository id="jobRepository" data-source="dataSource"
	    transaction-manager="transactionManager" isolation-level-for-create="READ_COMMITTED" 
	    table-prefix="BATCH_" max-varchar-length="1000"
	/>
	
	<bean:bean id="jobLauncher" 
		class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
		<bean:property name="jobRepository" ref="jobRepository"/>
	</bean:bean>
	
	<bean:bean id="transactionManager" 
			class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
		<bean:property name="dataSource" ref="dataSource" />
	</bean:bean>
	
    <bean:bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
  		<bean:property name="driverClassName">
   			<bean:value>${datasource.driver}</bean:value>
  		</bean:property>
  		<bean:property name="url">
   			<bean:value>${datasource.url}</bean:value>
  		</bean:property>
  		<bean:property name="username" value="${datasource.username}"></bean:property>
  		<bean:property name="password" value="${datasource.password}"></bean:property>
 	</bean:bean>
</bean:beans>

Batch的分區任務配置 job-partition-db.xml

<?xml version="1.0" encoding="UTF-8"?>
<bean:beans xmlns="http://www.springframework.org/schema/batch"    
    xmlns:bean="http://www.springframework.org/schema/beans" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    
    xmlns:p="http://www.springframework.org/schema/p" 
    xmlns:tx="http://www.springframework.org/schema/tx"    
    xmlns:aop="http://www.springframework.org/schema/aop" 
    xmlns:context="http://www.springframework.org/schema/context"    
    xsi:schemaLocation="http://www.springframework.org/schema/beans 
    http://www.springframework.org/schema/beans/spring-beans-4.0.xsd  
    http://www.springframework.org/schema/tx 
    http://www.springframework.org/schema/tx/spring-tx-4.0.xsd  
    http://www.springframework.org/schema/aop 
    http://www.springframework.org/schema/aop/spring-aop-4.0.xsd  
    http://www.springframework.org/schema/context 
    http://www.springframework.org/schema/context/spring-context-3.0.xsd
    http://www.springframework.org/schema/batch 
    http://www.springframework.org/schema/batch/spring-batch-3.0.xsd">
    <bean:import resource="classpath:db/job-context.xml"/>
    <bean:import resource="classpath:db/job-context-db.xml"/>
    
    <job id="partitionJob" restartable="true">
		<step id="partitionStep">
			<partition step="partitionReadWriteDB" partitioner="partitionerDB">
				<handler grid-size="4" task-executor="taskExecutor"/>
			</partition>
		</step>
	</job>
	
    <step id="partitionReadWriteDB">
		<tasklet >
			<chunk reader="jdbcItemPageReader" writer="jdbcItemWriter" 
			    processor="creditBillProcessor" commit-interval="2"/>
			 <listeners>
                  <listener ref="partitionItemReadListener"></listener>
             </listeners>
		</tasklet>
	</step>
	
    <!-- db數據切分 -->
	<bean:bean id="partitionerDB" 
	    class="com.demo.report.settle.DBpartition">
		<bean:property name="table" value="t_credit"/>
		<bean:property name="column" value="DID"/>
        <bean:property name="dataSource" ref="dataSource"/>
	</bean:bean>
	
	
	<!-- 從db分頁讀數據 -->
	<bean:bean id="jdbcItemPageReader" scope="step" 
	    class="org.springframework.batch.item.database.JdbcPagingItemReader">
        <bean:property name="dataSource" ref="dataSource"/>
        <bean:property name="queryProvider" ref="refQueryProvider" />
    	<bean:property name="pageSize" value="2"/>
        <bean:property name="rowMapper" ref="custCreditRowMapper"/>
    </bean:bean>
    
     <bean:bean id="refQueryProvider" scope="step" 
         class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
         <bean:property name="dataSource" ref="dataSource"/>
         <bean:property name="selectClause" value="select DID,ACCOUNTID,INNAME,AMOUNT,INDATE,ADDRESS"/>
         <bean:property name="fromClause" value="from t_credit"/>
         <bean:property name="whereClause" value="where MOD(DID,#{stepExecutionContext[_grideSize]}) = #{stepExecutionContext[_curPartition]} AND DID between #{stepExecutionContext[_minRecord]} and #{stepExecutionContext[_maxRecord]}"/>
         <bean:property name="sortKey" value="DID"/>
      </bean:bean>    
         
    <bean:bean id="custCreditRowMapper" 
        class="com.demo.report.settle.CreditBillRowMapper"/>
	
	<bean:bean id="jdbcItemWriter" 
        class="org.springframework.batch.item.database.JdbcBatchItemWriter">
        <bean:property name="dataSource" ref="dataSource"/>
        <bean:property name="sql" value="insert into t_destcredit (DID,ACCOUNTID,INNAME,AMOUNT,INDATE,ADDRESS) values (:id,:accountID,:name,:amount,:date,:address)"/>
        <bean:property name="itemSqlParameterSourceProvider">
            <bean:bean class="org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider"/>
        </bean:property>
    </bean:bean>
    
    <bean:bean id="creditBillProcessor" scope="step"
        class="com.demo.report.settle.CreditBillProcessor">
    </bean:bean>

    <bean:bean id="taskExecutor"
		class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
		<bean:property name="corePoolSize" value="5"/>
		<bean:property name="maxPoolSize" value="15"/>
	</bean:bean>
	
    <bean:bean id="partitionItemReadListener"
		class="com.demo.report.settle.PartitionStepExecutionListener">
	</bean:bean>
</bean:beans>

源碼

數據源PoJo-CreditBill.java

/**
 * 
 */
package com.demo.report.settle;

import java.io.Serializable;

/**
 * 信用卡對賬單模型.<br>
 * 
 * @author bruce.liu(mailto:[email protected])
 * 2014-1-11下午02:29:58
 */
public class CreditBill implements Serializable{
	/**
	 * 
	 */
	private static final long serialVersionUID = -4940717436114184875L;
	private String id;
	private String accountID = "";	/** 銀行卡賬戶ID */
	private String name = "";		/** 持卡人姓名 */
	private double amount = 0;		/** 消費金額 */
	private String date;			/** 消費日期 ，格式YYYY-MM-DD HH:MM:SS*/
	private String address;			/** 消費場所 **/
	
	public CreditBill(){}
	
	public CreditBill(String accountID, String name, double amount, String date, String address){
		this.accountID = accountID;
		this.name = name;
		this.amount = amount;
		this.date = date;
		this.address = address;
	}
	
	public String getId() {
		return id;
	}

	public void setId(String id) {
		this.id = id;
	}

	public String getAccountID() {
		return accountID;
	}
	public void setAccountID(String accountID) {
		this.accountID = accountID;
	}
	public String getName() {
		return name;
	}
	public void setName(String name) {
		this.name = name;
	}
	public double getAmount() {
		return amount;
	}
	public void setAmount(double amount) {
		this.amount = amount;
	}
	public String getDate() {
		return date;
	}
	public void setDate(String date) {
		this.date = date;
	}
	public String getAddress() {
		return address;
	}
	public void setAddress(String address) {
		this.address = address;
	}
	
	/**
	 * 
	 */
	public String toString(){
		StringBuffer sb = new StringBuffer();
		sb.append("id=" + getId() + ";accountID=" + getAccountID() + ";name=" + getName() + ";amount="
				+ getAmount() + ";date=" + getDate() + ";address=" + getAddress());
		return sb.toString();
	}
}

接收方PoJo DestinationCreditBill.java

/**
 * 
 */
package com.demo.report.settle;

import java.io.Serializable;

/**
 * 
 * @author bruce.liu(mailto:[email protected])
 * 2014-1-11下午02:37:55
 */
public class DestinationCreditBill implements Serializable{
	/**
	 * 
	 */
	private static final long serialVersionUID = 5253572139260172440L;
	private String id;
	private String accountID = "";	/** 銀行卡賬戶ID */
	private String name = "";		/** 持卡人姓名 */
	private double amount = 0;		/** 消費金額 */
	private String date;			/** 消費日期 ，格式YYYY-MM-DD HH:MM:SS*/
	private String address;			/** 消費場所 **/
	
	public DestinationCreditBill(){}
	
	public DestinationCreditBill(String accountID, String name, double amount, String date, String address){
		this.accountID = accountID;
		this.name = name;
		this.amount = amount;
		this.date = date;
		this.address = address;
	}
	
	public String getAccountID() {
		return accountID;
	}
	public void setAccountID(String accountID) {
		this.accountID = accountID;
	}
	public String getName() {
		return name;
	}
	public void setName(String name) {
		this.name = name;
	}
	public double getAmount() {
		return amount;
	}
	public void setAmount(double amount) {
		this.amount = amount;
	}
	public String getDate() {
		return date;
	}
	public void setDate(String date) {
		this.date = date;
	}
	public String getAddress() {
		return address;
	}
	public void setAddress(String address) {
		this.address = address;
	}
	
	public String getId() {
		return id;
	}

	public void setId(String id) {
		this.id = id;
	}

	/**
	 * 
	 */
	public String toString(){
		StringBuffer sb = new StringBuffer();
		sb.append("accountID=" + getAccountID() + ";name=" + getName() + ";amount="
				+ getAmount() + ";date=" + getDate() + ";address=" + getAddress());
		return sb.toString();
	}
}

分區步驟監視器 PartitionStepExecutionListener.java

/**
 * 
 */
package com.demo.report.settle;

import org.springframework.batch.core.ExitStatus;
import org.springframework.batch.core.StepExecution;
import org.springframework.batch.core.StepExecutionListener;


/**
 * 
 * @author bruce.liu(mailto:[email protected])
 * 2014-3-22上午10:11:29
 */
public class PartitionStepExecutionListener implements StepExecutionListener {
	@Override
	public void beforeStep(StepExecution stepExecution) {
		System.out.println("ThreadName=" + Thread.currentThread().getName() + "; " 
				+ "StepName=" + stepExecution.getStepName() + ";");
	}

	@Override
	public ExitStatus afterStep(StepExecution stepExecution) {
		return null;
	}
}

查詢結果集轉換器 CreditBillRowMapper.java

/**
 * 
 */
package com.demo.report.settle;

import java.sql.ResultSet;
import java.sql.SQLException;

import org.springframework.jdbc.core.RowMapper;

import com.demo.report.settle.CreditBill;


/**
 * 
 * @author bruce.liu(mailto:[email protected])
 * 2014-1-11下午03:10:28
 */
public class CreditBillRowMapper implements RowMapper<CreditBill> {

	public CreditBill mapRow(ResultSet rs, int rowNum) throws SQLException {
		CreditBill bill = new CreditBill();
		bill.setId(rs.getString("DID"));
		bill.setAccountID(rs.getString("ACCOUNTID"));
		bill.setAddress(rs.getString("ADDRESS"));
		bill.setAmount(rs.getDouble("AMOUNT"));
		bill.setDate(rs.getString("INDATE"));
		bill.setName(rs.getString("INNAME"));
		return bill;
	}
}

分區策略生成器 DBpartition.java

package com.demo.report.settle;

import java.text.MessageFormat;
import java.util.HashMap;
import java.util.Map;

import javax.sql.DataSource;

import org.springframework.batch.core.partition.support.Partitioner;
import org.springframework.batch.item.ExecutionContext;
import org.springframework.jdbc.core.JdbcTemplate;

/**
 * 
 * @author bruce.liu(mailto:[email protected])
 * 2014-1-11下午03:13:45
 */
public class DBpartition implements Partitioner {
	private static final String _MINRECORD = "_minRecord";
	private static final String _MAXRECORD = "_maxRecord";
	private static final String _GRIDSIZE = "_grideSize";
	private static final String _CURPARTITION = "_curPartition";
	private static final String MIN_SELECT_PATTERN = "select min({0}) from {1}";
	private static final String MAX_SELECT_PATTERN = "select max({0}) from {1}";
	private JdbcTemplate jdbcTemplate ;
	private DataSource dataSource;
	private String table ;
	private String column;
	public Map<String, ExecutionContext> partition(int gridSize) {
		validateAndInit();
		Map<String, ExecutionContext> resultMap = new HashMap<String, ExecutionContext>();
		int min = jdbcTemplate.queryForObject(MessageFormat.format(MIN_SELECT_PATTERN, new Object[]{column,table}),Integer.class);
		int max = jdbcTemplate.queryForObject(MessageFormat.format(MAX_SELECT_PATTERN, new Object[]{column,table}),Integer.class);
		int targetSize = (max-min)/gridSize +1;
		int number=0;
		int start =min;
		int end = start+targetSize-1;
		
		while(start <= max){
			ExecutionContext context = new ExecutionContext();
			if(end>=max){
				end=max;
			}
			context.putInt(_MINRECORD, start);
			context.putInt(_MAXRECORD, end);
			context.putLong(_CURPARTITION, Thread.currentThread().getId());
			context.putInt(_GRIDSIZE, gridSize);
			start+=targetSize;
			end+=targetSize;
			resultMap.put("partition"+(number++), context);
		}
		System.out.println("gridSize is ="+gridSize+"***current thread id is "+Thread.currentThread().getId()+"*** name  is "+Thread.currentThread().getName());
		if(!resultMap.isEmpty())
		{
			resultMap.forEach((k,v)->{System.out.println("key is "+k+"****values is"+v);});
			
		}
		return resultMap;
	}
	
	public void validateAndInit(){
		if(isEmpty(table)){
			throw new IllegalArgumentException("table cannot be null");
		}
		if(isEmpty(column)){
			throw new IllegalArgumentException("column cannot be null");
		}
		if(dataSource!=null && jdbcTemplate==null){
			jdbcTemplate = new JdbcTemplate(dataSource);
		}
		if(jdbcTemplate==null){
			throw new IllegalArgumentException("jdbcTemplate cannot be null");
		}
	}

	public static boolean isEmpty(String info){
		if(info!=null){
			if(info.trim().length()>1){
				return false;
			}
		}
		return true;
	}
	
	public String getColumn() {
		return column;
	}

	public void setColumn(String column) {
		this.column = column;
	}

	public String getTable() {
		return table;
	}

	public void setTable(String table) {
		this.table = table;
	}

	public JdbcTemplate getJdbcTemplate() {
		return jdbcTemplate;
	}

	public void setJdbcTemplate(JdbcTemplate jdbcTemplate) {
		this.jdbcTemplate = jdbcTemplate;
	}

	public DataSource getDataSource() {
		return dataSource;
	}

	public void setDataSource(DataSource dataSource) {
		this.dataSource = dataSource;
	}
}

分區處理器 CreditBillProcessor.java

/**
 * 
 */
package com.demo.report.settle;

import org.springframework.batch.item.ItemProcessor;

/**
 * 
 * @author bruce.liu(mailto:[email protected])
 * 2014-1-11下午02:38:01
 */
public class CreditBillProcessor implements
		ItemProcessor<CreditBill, DestinationCreditBill> {

	public DestinationCreditBill process(CreditBill bill) throws Exception {
		System.out.println(bill.toString());
		DestinationCreditBill destCreditBill = new DestinationCreditBill();
		destCreditBill.setAccountID(bill.getAccountID());
		destCreditBill.setAddress(bill.getAddress());
		destCreditBill.setAmount(bill.getAmount());
		destCreditBill.setDate(bill.getDate());
		destCreditBill.setId(bill.getId());
		destCreditBill.setName(bill.getName());
		return destCreditBill;
	}
}

測試類基類 JobLaunchBase.java

package com.snail.report.settle;


import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParametersBuilder;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
/**
 * 
 */

/**
 * @author bruce.liu(mailto:[email protected])
 * 2013-9-7下午01:57:02
 */
public class JobLaunchBase {
	private static ApplicationContext context;

	/**
	 * 執行批處理作業.<br>
	 * @param jobPath	作業配置文件
	 * @param jobName	作業名
	 * @param builder	作業參數構造器
	 */
	public static void executeJob(String jobPath, String jobName, JobParametersBuilder builder) {
		context = new ClassPathXmlApplicationContext(jobPath);
		JobLauncher launcher = (JobLauncher) context.getBean("jobLauncher");
		Job job = (Job) context.getBean(jobName);
		try {
			JobExecution result = launcher.run(job, builder.toJobParameters());
			System.out.println(result.toString());
		} catch (Exception e) {
			e.printStackTrace();
		}finally {
			context=null;
			job=null;
			builder=null;
			launcher=null;
		}
		
	}
}

測試類 JobLaunchPartitionDBOracle.java

package com.snail.report.settle;
/**
 * 
 */


import java.util.Date;

import org.springframework.batch.core.JobParametersBuilder;

import com.snail.report.settle.JobLaunchBase;

/**
 * 
 * @author bruce.liu(mailto:[email protected])
 * 2013-11-16下午10:59:46
 */
public class JobLaunchPartitionDBOracle {
	
	/**
	 * @param args
	 */
	public static void main(String[] args) {
		JobLaunchBase.executeJob("db/job-partition-db.xml", "partitionJob",
				new JobParametersBuilder().addDate("date", new Date()));
		System.exit(0);
	}
}

問題總結

找不到batch

原因是創建spring boot項目時忘記加batch了


		<spring.batch.version>3.2.1.RELEASE</spring.batch.version>

		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-core</artifactId>
			<version>${spring.batch.version}</version>
			<type>jar</type>
			<optional>false</optional>
		</dependency>
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-infrastructure</artifactId>
			<version>${spring.batch.version}</version>
			<type>jar</type>
			<optional>false</optional>
		</dependency>
		<dependency>
			<groupId>org.springframework.batch</groupId>
			<artifactId>spring-batch-integration</artifactId>
			<version>1.2.0.RELEASE</version>
			<optional>false</optional>
		</dependency>

找不到oracle驅動

手工加入maven倉庫

mvn install:install-file -DgroupId=com.oracle -DartifactId=ojdbc14 -Dversion=10.2.0.1.0 -Dpackaging=jar -Dfile=ojdbc14-10.2.0.1.0.jar

無法連續事務處理

ORA-08177: 無法連續訪問此事務處理 batch
org.springframework.context.support.AbstractRefreshableApplicationContext.refreshBeanFactory(AbstractRefreshableApplicationContext.java:131)
	at org.springframework.context.support.AbstractApplicationContext.obtainFreshBeanFactory(AbstractApplicationContext.java:537)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:451)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)

解決辦法 job-repository 改事務級別

	<!--  原來是這個 "SERIALIZABLE" -->	
	<job-repository id="jobRepository" data-source="dataSource"
	    transaction-manager="transactionManager" isolation-level-for-create="READ_COMMITTED" 
	    table-prefix="BATCH_" max-varchar-length="1000"
	/>

jdbcTemplate找不到 queryForInt 方法

The method queryForInt(String) is undefined for the type JdbcTemplate	DBpartition.java

因爲使用spring4，默認jdbc版本也高了，原有的方法沒了，所以解決辦法是改成 jdbcTemplate.queryForObject 注意要新加一個參數是 返回對象的類型

		int max = jdbcTemplate.queryForObject(MessageFormat.format(MAX_SELECT_PATTERN, new Object[]{column,table}),**Integer.class**);

加了mod，細化分區

加這個算是一點試驗，剛開始沒注意到變量名的問題，執行錯誤，後來發現
配置文件中的where條件如果取變量值 ，需要stepExecutionContext+變量名 纔可以
參考我加的**_grideSize**就明白了。

總結

總體來說改造難度不大，下一次版本應該是註解版的了，或許挑戰一下分區多線程。
推薦文章
http://www.kailing.pub/article/index/arcid/196.html
https://www.cnblogs.com/softidea/p/7142084.html

發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

Spring Cloud Ribbon負載均衡策略詳解

Spring Cloud Ribbon負載均衡策略詳解。 Spring Cloud Ribbon是一個基於HTTP和TCP的客戶端負載均衡工具，它基於Netflix Ribbon實現，Spring Cloud集成了Netflix Ribb

春秋战国程序猿 2020-07-08 00:17:06

微服務實戰（一） 如何理解SpringBoot 、SpringCloud、Docker、K8s

本篇主要介紹對於微服務的理解 現如今，隨着互聯網的發展，對於系統性能，架構均有了更高的要求。以前傳統的ssh時代的單體應用的機構模式已顯力不從心，所以微服務架構應運而生，並且形成了越來越成熟的方案。寫本系列文章的目標是從單體架構模式到微服

sinao139 2020-07-08 12:13:57

使用Jenkins編譯打包SpringCloud微服務中的個別目錄

使用Jenkins編譯打包SpringCloud微服務中的個別目錄

超级大超越 2020-07-08 09:04:40

Idea+springcloud+zookeeper做的服務發現和遠程連接zookeeper

idea+springcloud+zookeeper做的服務發現和遠程連接zookeeper

超级大超越 2020-07-08 09:04:39

通過docker，部署SpringCloud項目

通過docker，部署SpringCloud項目

超级大超越 2020-07-08 09:04:28

Feign 調用文件服務上傳文件

feign文件上傳報錯FileUploadException: the request was rejected because no multipart boundary feign-form和spring cloud版本不一致

换个新手机 2020-07-08 08:47:57

Spring Cloud Alibaba與spring Cloud系列之 Dubbo與Spring Cloud Alibaba

參考文獻：http://blog.didispace.com/spring-cloud-alibaba-dubbo-1/      之前爲了解Dubbo與spring Cloud之間的關係和區別，看過很多Spring Cloud與Dubb

仙羽刺士 2020-07-08 07:37:02

綜合網上博客，最終版終結：spring cloud Feign 上傳文件 ！！

    這兩天在做一個 Feign 上傳文件的需求，spring cloud官網上說 Feign 不支持 form 表單上傳文件，所以我去了各大博客網站，綜合看了很多相關的博客，走了很多彎路，也跳了很多坑，還好最終實現了自己的需求，所以在

TianXieZuoMaiKong 2020-07-08 00:32:18

大話微服務配置中心：Eureka Server與Eureka Client的契約（Lease.class）

Eureka Server與Eureka Client之間簽訂了一個契約（Lease.class），這是它們之間正常運轉的根基。 寒來暑往，年復一年，由於契約的存在，微服務系統得以提供穩定的發現註冊服務。 以心跳續約爲例，Eureka C

春秋战国程序猿 2020-07-08 00:17:07

自己動手，使用Spring Initializr從零開始搭建Spring Cloud項目

  新建Project   這裏使用的開發工具是IDEA，JDK版本1.8。 打開IDEA開發工具，File -> New -> Project    然後一步步往下設置，然後到這一步，選擇Spring Cloud   OK，繼續一路

春秋战国程序猿 2020-07-08 00:17:06

Spring Cloud如何優雅打印日誌：slf4j+logback

最近在對項目的日誌進行優化，主要是如何減少不必要的日誌輸出，如何優化日誌輸出的性能， 以及當前code中，一些不規範的日誌輸出代碼的優化。基於此，對java日誌進行了一個系統的梳理。 今天這裏，主要分享一個點，乾貨！具體的理論就不再重複搬

春秋战国程序猿 2020-07-08 00:17:05

springBoot 項目自定義命名application.properties配置文件名稱

最近因項目原因，需要將springboot項目的properties文件改爲自定義名稱，在查看一些資料後最終解決，特記錄一下！ 參考資料：https://docs.spring.io/spring-boot/docs/current/re

ywb201314 2020-07-07 19:54:57

Spring Cloud 微服務學習筆記04 服務治理 Zookeeper基礎

1.   Zookeeper     1.1 Zookeeper 概述     Zookeeper是應用於分佈式應用程序的高性能分佈式協調服務，它暴露了一組簡單的公共服務（提供java和C接口），如命名、配置管理、集羣服務、分佈式鎖等，分

lbmydream 2020-07-07 18:56:03

SpringCloud微服務架構實戰-微服務商城 （2） 註冊中心搭建

1、服務治理 服務治理是微服務架構的最核心和基礎模塊，用於實現各個微服務間的自動化註冊和發現。服務治理解決了微服務系統架構中微服務實例配置維護困難問題。服務治理框架圍繞服務註冊和服務發現機制來完成對微服務應用實例的自動化管理 2、 服務註

lbmydream 2020-07-07 17:32:40

《微服務框架之Spring Cloud》一、簡介

本文中我們主要介紹微服務開發框架——Spring Cloud。儘管Spring Cloud帶有"Cloud"的字樣，但它並不是雲計算解決方案，而是Spring Boot的基礎上構建的，用於快速構建分佈式系統的通用模式的工具集。 Sprin

FlyMarion 2020-07-07 15:36:19