簡介
Quartz是一個完全由java編寫的開源作業調度框架。不要讓作業調度這個術語嚇着你。儘管Quartz框架整合了許多額外功能, 但就其簡易形式看,你會發現它易用得簡直讓人受不了!。簡單地創建一個實現org.quartz.Job接口的java類。Job接口包含唯一的方法:
public void execute(JobExecutionContext context) throws JobExecutionException;
在你的Job接口實現類裏面,添加一些邏輯到execute()方法。一旦你配置好Job實現類並設定好調度時間表,Quartz將密切注意剩餘時間。當調度程序確定該是通知你的作業的時候,Quartz框架將調用你Job實現類(作業類)上的execute()方法並允許做它該做的事情。無需報告任何東西給調度器或調用任何特定的東西。僅僅執行任務和結束任務即可。如果配置你的作業在隨後再次被調用,Quartz框架將在恰當的時間再次調用它。
單機部署問題
單機模式下的定時任務調用很簡單,有很多可實現的方案,這裏不多說了,例如spring schedule,java timer等。
這裏說一下集羣部署的情況下,定時任務的使用。這種情況下,quartz是一個比較好的選擇。簡單,穩定。
想象一下,現在有 A , B , C 3 臺機器同時作爲集羣服務器對外統一提供 SERVICE :
A , B , C 3 臺機器上各有一個 QUARTZ Job,它們會按照即定的 SCHEDULE 自動執行各自的任務。
先不說實現什麼功能,這樣的架構有點像多線程。由於三臺SERVER 裏都有 QUARTZ ,因此會存在重複處理 TASK 的現象。
一般外面的解決方案是隻在一臺 服務器上裝 QUARTZ ,其它兩臺不裝,這樣的話其實就是單機了,quartz存在單點問題,一旦裝有quartz的服務器宕了。服務就無法提供了。
當然還有其他一些解決方案,無非就是改 QUARTZ JOB 的代碼了,這對程序開發人員來說比較痛苦;
而quartz本身提供了很好的集羣方案。下面我們來說一下在spring boot下的集成:
quartz集羣需要數據庫的支持(JobStore TX或者JobStoreCMT),從本質上來說,是使集羣上的每一個節點通過共享同一個數據庫來工作的
1 準備quartz基本環境
到quartz官網下載最新的包:http://www.quartz-scheduler.org/downloads/
解壓後,可以看到結構目錄。在\docs\dbTables下選擇合適你數據庫的SQL執行文件,創建quartz集羣需要的表(共11張表)
找到自己使用的數據庫腳本文件執行
數據庫中對應表,注意:在windows環境下,mysql表名不確認大小寫,linux下區分大小寫
2 集成springboot(這裏是1.5.3版本)
2.1引入依賴jar包
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz</artifactId>
<version>2.2.3</version>
</dependency>
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz-jobs</artifactId>
<version>2.2.3</version>
</dependency>
2.2 quartz配置
#quartz集羣配置
# ===========================================================================
# Configure Main Scheduler Properties 調度器屬性
# ===========================================================================
#調度標識名 集羣中每一個實例都必須使用相同的名稱
org.quartz.scheduler.instanceName=DefaultQuartzScheduler
#ID設置爲自動獲取 每一個必須不同
org.quartz.scheduler.instanceid=AUTO
#============================================================================
# Configure ThreadPool
#============================================================================
#線程池的實現類(一般使用SimpleThreadPool即可滿足幾乎所有用戶的需求)
org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
#指定線程數,至少爲1(無默認值)(一般設置爲1-100直接的整數合適)
org.quartz.threadPool.threadCount = 25
#設置線程的優先級(最大爲java.lang.Thread.MAX_PRIORITY 10,最小爲Thread.MIN_PRIORITY 1,默認爲5)
org.quartz.threadPool.threadPriority = 5
#============================================================================
# Configure JobStore
#============================================================================
# 信息保存時間 默認值60秒
org.quartz.jobStore.misfireThreshold = 60000
#數據保存方式爲數據庫持久化
org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
#數據庫代理類,一般org.quartz.impl.jdbcjobstore.StdJDBCDelegate可以滿足大部分數據庫
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate
#JobDataMaps是否都爲String類型
org.quartz.jobStore.useProperties = false
#數據庫別名 隨便取
org.quartz.jobStore.dataSource = myDS
#表的前綴,默認QRTZ_
org.quartz.jobStore.tablePrefix = QRTZ_
#是否加入集羣
org.quartz.jobStore.isClustered = true
#調度實例失效的檢查時間間隔
org.quartz.jobStore.clusterCheckinInterval = 20000
#============================================================================
# Configure Datasources
#============================================================================
#數據庫引擎
org.quartz.dataSource.myDS.driver = com.mysql.jdbc.Driver
#數據庫連接
org.quartz.dataSource.myDS.URL = jdbc:mysql://172.30.12.14:7001/rbl_test?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowMultiQueries=true
#數據庫用戶
org.quartz.dataSource.myDS.user = root
#數據庫密碼
org.quartz.dataSource.myDS.password = 123456
#允許最大連接
org.quartz.dataSource.myDS.maxConnections = 5
#驗證查詢sql,可以不設置
org.quartz.dataSource.myDS.validationQuery=select 0 from dual
注:如果嫌需要額外配置quart數據源很煩,也可以共用你項目配置的數據庫鏈接,這樣每次更換數據庫連接,就不需要額外在修改。
2.3 springboot configuration 配置
直接使用springboot注入的的datasource的內容配置quartz的數據庫連接
quartz屬性配置可以讀取配置文件讀取,我這裏沒在配置文件讀取,直接寫在代碼裏測試的,意見新建一個配置文件裏面寫quartz的配置內容,然後通過springboot注入屬性進來。
package com.kerry.config;
import java.io.IOException;
import java.util.Properties;
import javax.sql.DataSource;
import org.quartz.Scheduler;
import org.quartz.SchedulerException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.quartz.SchedulerFactoryBean;
/**
* 分佈式定時任務管理配置
* @author kerry
* @date 2018-05-09 11:36:21
*/
@Configuration
//@ConditionalOnProperty(prefix = "qybd", name = "quartz-open", havingValue = "true")
public class QuartzConfig{
@Autowired
DataSource dataSource;
@Bean
public SchedulerFactoryBean schedulerFactoryBean(QuartzJobFactory myJobFactory) throws Exception {
SchedulerFactoryBean schedulerFactoryBean = new SchedulerFactoryBean();
schedulerFactoryBean.setDataSource(dataSource);
//使job實例支持spring 容器管理
schedulerFactoryBean.setOverwriteExistingJobs(true);
schedulerFactoryBean.setJobFactory(myJobFactory);
schedulerFactoryBean.setQuartzProperties(quartzProperties());
// 延遲10s啓動quartz
schedulerFactoryBean.setStartupDelay(10);
return schedulerFactoryBean;
}
@Bean
public Scheduler scheduler(SchedulerFactoryBean schedulerFactoryBean) throws IOException, SchedulerException {
// SchedulerFactory schedulerFactory = new StdSchedulerFactory(quartzProperties());
// Scheduler scheduler = schedulerFactory.getScheduler();
// scheduler.start();//初始化bean並啓動scheduler
Scheduler scheduler = schedulerFactoryBean.getScheduler();
scheduler.start();
return scheduler;
}
/**
* 設置quartz屬性
*/
public Properties quartzProperties() throws IOException {
Properties prop = new Properties();
prop.put("quartz.scheduler.instanceName", "ServerScheduler");
prop.put("org.quartz.scheduler.instanceId", "AUTO");
prop.put("org.quartz.scheduler.skipUpdateCheck", "true");
prop.put("org.quartz.scheduler.instanceId", "NON_CLUSTERED");
prop.put("org.quartz.scheduler.jobFactory.class", "org.quartz.simpl.SimpleJobFactory");
prop.put("org.quartz.jobStore.class", "org.quartz.impl.jdbcjobstore.JobStoreTX");
prop.put("org.quartz.jobStore.driverDelegateClass", "org.quartz.impl.jdbcjobstore.StdJDBCDelegate");
prop.put("org.quartz.jobStore.dataSource", "quartzDataSource");
prop.put("org.quartz.jobStore.tablePrefix", "QRTZ_");
prop.put("org.quartz.jobStore.isClustered", "true");
prop.put("org.quartz.threadPool.class", "org.quartz.simpl.SimpleThreadPool");
prop.put("org.quartz.threadPool.threadCount", "5");
// prop.put("org.quartz.dataSource.quartzDataSource.driver", druidProperties.getDriverClassName());
// prop.put("org.quartz.dataSource.quartzDataSource.URL", druidProperties.getUrl());
// prop.put("org.quartz.dataSource.quartzDataSource.user", druidProperties.getUsername());
// prop.put("org.quartz.dataSource.quartzDataSource.password", druidProperties.getPassword());
// prop.put("org.quartz.dataSource.quartzDataSource.maxConnections", druidProperties.getMaxActive());
return prop;
}
}
注意上面的schedulerFactoryBean.setJobFactory(myJobFactory); //這個myJobFactory是自定義配置的一個類,如果這裏不配置這個jobFactory,下面的那個CtripScenicTask會爲空,獲取不了注入對象
@Component
public class CtripScenicJob implements Job{
private Logger logger = LoggerFactory.getLogger(CtripScenicJob.class);
@Autowired
private CtripScenicTask ctripScenicTask;
這個類主要解決spring管理的Quartz job裏面注入不了其他bean
package com.kerry.config;
import org.quartz.spi.TriggerFiredBundle;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.config.AutowireCapableBeanFactory;
import org.springframework.scheduling.quartz.AdaptableJobFactory;
import org.springframework.stereotype.Component;
@Component
public class QuartzJobFactory extends AdaptableJobFactory {
//這個對象Spring會幫我們自動注入進來,也屬於Spring技術範疇.
@Autowired
private AutowireCapableBeanFactory capableBeanFactory;
protected Object createJobInstance(TriggerFiredBundle bundle) throws Exception {
//調用父類的方法
Object jobInstance = super.createJobInstance(bundle);
//進行注入,這屬於Spring的技術,不清楚的可以查看Spring的API.
capableBeanFactory.autowireBean(jobInstance);
return jobInstance;
}
}
2.4 動態配置管理quartz
接口類
package com.kerry.modular.biz.service;
import java.util.List;
import org.quartz.SchedulerException;
import com.kerry.modular.biz.model.TaskInfo;
public interface TaskService {
List<TaskInfo> list();
void addJob(TaskInfo info);
void edit(TaskInfo info);
void delete(String jobName, String jobGroup);
void pause(String jobName, String jobGroup);
void resume(String jobName, String jobGroup);
boolean checkExists(String jobName, String jobGroup)throws SchedulerException;
}
實現類
package com.kerry.modular.biz.service;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashSet;
import java.util.List;
import org.apache.commons.lang3.time.DateFormatUtils;
import org.quartz.CronScheduleBuilder;
import org.quartz.CronTrigger;
import org.quartz.Job;
import org.quartz.JobBuilder;
import org.quartz.JobDetail;
import org.quartz.JobKey;
import org.quartz.Scheduler;
import org.quartz.SchedulerException;
import org.quartz.Trigger;
import org.quartz.TriggerBuilder;
import org.quartz.TriggerKey;
import org.quartz.impl.matchers.GroupMatcher;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import com.kerry.modular.biz.model.TaskInfo;
@Service
public class TaskServiceImpl implements TaskService {
private Logger logger = LoggerFactory.getLogger(TaskServiceImpl.class);
@Autowired(required=false)
private Scheduler scheduler;
/**
* 所有任務列表
*/
public List<TaskInfo> list(){
List<TaskInfo> list = new ArrayList<>();
try {
for(String groupJob: scheduler.getJobGroupNames()){
for(JobKey jobKey: scheduler.getJobKeys(GroupMatcher.<JobKey>groupEquals(groupJob))){
List<? extends Trigger> triggers = scheduler.getTriggersOfJob(jobKey);
for (Trigger trigger: triggers) {
Trigger.TriggerState triggerState = scheduler.getTriggerState(trigger.getKey());
JobDetail jobDetail = scheduler.getJobDetail(jobKey);
String cronExpression = "", createTime = "";
if (trigger instanceof CronTrigger) {
CronTrigger cronTrigger = (CronTrigger) trigger;
cronExpression = cronTrigger.getCronExpression();
createTime = cronTrigger.getDescription();
}
TaskInfo info = new TaskInfo();
info.setJobName(jobKey.getName());
info.setJobGroup(jobKey.getGroup());
info.setJobDescription(jobDetail.getDescription());
info.setJobStatus(triggerState.name());
info.setCronExpression(cronExpression);
info.setCreateTime(createTime);
list.add(info);
}
}
}
} catch (SchedulerException e) {
e.printStackTrace();
}
return list;
}
/**
* 保存定時任務
* @param info
*/
@SuppressWarnings("unchecked")
public void addJob(TaskInfo info) {
String jobName = info.getJobName(),
jobGroup = info.getJobGroup(),
cronExpression = info.getCronExpression(),
jobDescription = info.getJobDescription(),
createTime = DateFormatUtils.format(new Date(), "yyyy-MM-dd HH:mm:ss");
try {
if (checkExists(jobName, jobGroup)) {
logger.info("add job fail, job already exist, jobGroup:{}, jobName:{}", jobGroup, jobName);
}
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
JobKey jobKey = JobKey.jobKey(jobName, jobGroup);
CronScheduleBuilder schedBuilder = CronScheduleBuilder.cronSchedule(cronExpression).withMisfireHandlingInstructionDoNothing();
CronTrigger trigger = TriggerBuilder.newTrigger().withIdentity(triggerKey).withDescription(createTime).withSchedule(schedBuilder).build();
Class<? extends Job> clazz = (Class<? extends Job>)Class.forName(jobName);
JobDetail jobDetail = JobBuilder.newJob(clazz).withIdentity(jobKey).withDescription(jobDescription).build();
scheduler.scheduleJob(jobDetail, trigger);
} catch (SchedulerException | ClassNotFoundException e) {
logger.error("類名不存在或執行表達式錯誤,exception:{}",e.getMessage());
}
}
/**
* 修改定時任務
* @param info
*/
public void edit(TaskInfo info) {
String jobName = info.getJobName(),
jobGroup = info.getJobGroup(),
cronExpression = info.getCronExpression(),
jobDescription = info.getJobDescription(),
createTime = DateFormatUtils.format(new Date(), "yyyy-MM-dd HH:mm:ss");
try {
if (!checkExists(jobName, jobGroup)) {
logger.info("edit job fail, job is not exist, jobGroup:{}, jobName:{}", jobGroup, jobName);
}
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
JobKey jobKey = new JobKey(jobName, jobGroup);
CronScheduleBuilder cronScheduleBuilder = CronScheduleBuilder.cronSchedule(cronExpression).withMisfireHandlingInstructionDoNothing();
CronTrigger cronTrigger = TriggerBuilder.newTrigger().withIdentity(triggerKey).withDescription(createTime).withSchedule(cronScheduleBuilder).build();
JobDetail jobDetail = scheduler.getJobDetail(jobKey);
jobDetail.getJobBuilder().withDescription(jobDescription);
HashSet<Trigger> triggerSet = new HashSet<>();
triggerSet.add(cronTrigger);
scheduler.scheduleJob(jobDetail, triggerSet, true);
} catch (SchedulerException e) {
logger.error("類名不存在或執行表達式錯誤,exception:{}",e.getMessage());
}
}
/**
* 刪除定時任務
* @param jobName
* @param jobGroup
*/
public void delete(String jobName, String jobGroup){
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
try {
if (checkExists(jobName, jobGroup)) {
scheduler.pauseTrigger(triggerKey);
scheduler.unscheduleJob(triggerKey);
logger.info("delete job, triggerKey:{},jobGroup:{}, jobName:{}", triggerKey ,jobGroup, jobName);
}
} catch (SchedulerException e) {
logger.error(e.getMessage());
}
}
/**
* 暫停定時任務
* @param jobName
* @param jobGroup
*/
public void pause(String jobName, String jobGroup){
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
try {
if (checkExists(jobName, jobGroup)) {
scheduler.pauseTrigger(triggerKey);
logger.info("pause job success, triggerKey:{},jobGroup:{}, jobName:{}", triggerKey ,jobGroup, jobName);
}
} catch (SchedulerException e) {
logger.error(e.getMessage());
}
}
/**
* 重新開始任務
* @param jobName
* @param jobGroup
*/
public void resume(String jobName, String jobGroup){
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
try {
if (checkExists(jobName, jobGroup)) {
scheduler.resumeTrigger(triggerKey);
logger.info("resume job success,triggerKey:{},jobGroup:{}, jobName:{}", triggerKey ,jobGroup, jobName);
}
} catch (SchedulerException e) {
logger.error(e.getMessage());
}
}
/**
* 驗證是否存在
* @param jobName
* @param jobGroup
* @throws SchedulerException
*/
public boolean checkExists(String jobName, String jobGroup) throws SchedulerException{
TriggerKey triggerKey = TriggerKey.triggerKey(jobName, jobGroup);
return scheduler.checkExists(triggerKey);
}
}
taskinfo實體類
package com.kerry.modular.biz.model;
import java.io.Serializable;
public class TaskInfo implements Serializable{
private static final long serialVersionUID = -8054692082716173379L;
private int id = 0;
/**任務名稱*/
private String jobName;
/**任務分組*/
private String jobGroup;
/**任務描述*/
private String jobDescription;
/**任務狀態*/
private String jobStatus;
/**任務表達式*/
private String cronExpression;
private String createTime;
public String getJobName() {
return jobName;
}
public void setJobName(String jobName) {
this.jobName = jobName;
}
public String getJobGroup() {
return jobGroup;
}
public void setJobGroup(String jobGroup) {
this.jobGroup = jobGroup;
}
public String getJobDescription() {
return jobDescription;
}
public void setJobDescription(String jobDescription) {
this.jobDescription = jobDescription;
}
public String getJobStatus() {
return jobStatus;
}
public void setJobStatus(String jobStatus) {
this.jobStatus = jobStatus;
}
public String getCronExpression() {
return cronExpression;
}
public void setCronExpression(String cronExpression) {
this.cronExpression = cronExpression;
}
public String getCreateTime() {
return createTime;
}
public void setCreateTime(String createTime) {
this.createTime = createTime;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
}
任務管理Controller類
package com.kerry.modular.biz.controller;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import com.alibaba.fastjson.JSON;
import com.kerry.modular.biz.model.TaskInfo;
import com.kerry.modular.biz.service.TaskService;
/**
* 任務管理
*/
@Controller
@RequestMapping("/qy/api/task/")
public class TaskManageController {
@Autowired(required=false)
private TaskService taskService;
/**
* Index.jsp
*/
@RequestMapping(value={"", "/", "index"})
public String info(){
return "index.jsp";
}
/**
* 任務列表
* @return
*/
@ResponseBody
@RequestMapping(value="list")
public String list(){
Map<String, Object> map = new HashMap<>();
List<TaskInfo> infos = taskService.list();
map.put("rows", infos);
map.put("total", infos.size());
return JSON.toJSONString(map);
}
/**
* 保存定時任務
* @param info
*/
@ResponseBody
@RequestMapping(value="save", produces = "application/json; charset=UTF-8")
public String save(TaskInfo info){
try {
if(info.getId() == 0) {
taskService.addJob(info);
}else{
taskService.edit(info);
}
} catch (Exception e) {
return e.getMessage();
}
return "成功";
}
/**
* 刪除定時任務
* @param jobName
* @param jobGroup
*/
@ResponseBody
@RequestMapping(value="delete/{jobName}/{jobGroup}", produces = "application/json; charset=UTF-8")
public String delete(@PathVariable String jobName, @PathVariable String jobGroup){
try {
taskService.delete(jobName, jobGroup);
} catch (Exception e) {
return e.getMessage();
}
return "成功";
}
/**
* 暫停定時任務
* @param jobName
* @param jobGroup
*/
@ResponseBody
@RequestMapping(value="pause/{jobName}/{jobGroup}", produces = "application/json; charset=UTF-8")
public String pause(@PathVariable String jobName, @PathVariable String jobGroup){
try {
taskService.pause(jobName, jobGroup);
} catch (Exception e) {
return e.getMessage();
}
return "成功";
}
/**
* 重新開始定時任務
* @param jobName
* @param jobGroup
*/
@ResponseBody
@RequestMapping(value="resume/{jobName}/{jobGroup}", produces = "application/json; charset=UTF-8")
public String resume(@PathVariable String jobName, @PathVariable String jobGroup){
try {
taskService.resume(jobName, jobGroup);
} catch (Exception e) {
return e.getMessage();
}
return "成功";
}
}
任務實現類實現job接口,重寫execute方法
package com.kerry.modular.biz.task.quartz;
import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import com.kerry.modular.biz.task.schedule.CtripScenicTask;
@Component
public class CtripScenicJob implements Job{
private Logger logger = LoggerFactory.getLogger(CtripScenicJob.class);
@Autowired
private CtripScenicTask ctripScenicTask;
@Override
public void execute(JobExecutionContext context)
throws JobExecutionException {
logger.info("JobName: {}", context.getJobDetail().getKey().getName());
ctripScenicTask.loadComment();
}
}
此時可以通過調用TaskManageController時間動態控制定時任務
3 測試,啓動springboot項目
輸入添加任務的url:
http://localhost:8080/qy/api/task/save?jobName=com.stylefeng.guns.modular.biz.task.quartz.CtripHotelJob&jobGroup=group1&jobDescription=job描述&cronExpression=0/10 * * * * ?
jobName爲job類的包名類名,jobGroup該任務所屬組,jobDescription 描述,cronExpression :core表達式
上面的請求會添加一個定時任務,每10秒執行一次 CtripHotelJob裏面的execute方法。
保存的定時任務會在quartz相關表裏保存數據
如: