整合Elastic-Job(支持动态任务)

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 最近公司的项目需要用到分布式任务调度，在结合多款开源框架后决定使用当当的Elastic-job。不知道大家有没有这样的需求，就是动态任务。之前比较了xxl-job和elastic-job发现，都只是支持注解或者配置以及后台添加现有的任务，不支持动态添加。比如：类似订单半小时后自动取消的场景。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" xxl-job理论上来说是可以支持的，但是需要高度整合admin端的程序，然后开放对应的接口才可以给其他服务调用，这样本质直接改源码对后期的升级十分不便，最后放弃了xxl-job。elastic-job在移交Apache后的版本规划中，有提到API的开放，但是目前还没有稳定版，所以只能使用之前的2.1.5的版本来做。在Github搜了很多整合方案，最后决定选择下面的来实现。"}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"\n com.github.xjzrc.spring.boot\n elastic-job-lite-spring-boot-starter\n ${lasted.release.version}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5c/5c71947a0cced6a7664f6dfe6832e8de.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 因为要做的是动态的，所以这里没有直接使用maven座标引入，直接将源码全部接入项目来使用，这样比较灵活，因为底层本质上还是用elastic-job的东西。下面引入elastic-job座标"}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":" \n \t\tcom.dangdang\n \t\telastic-job-lite-spring\n \t\t${elastic-job.version}\n \n \n \t\tcom.dangdang\n \t\telastic-job-lite-lifecycle\n \t\t${elastic-job.version}\n \n \n org.eclipse.jetty.aggregate\n jetty-all-server\n \n \n org.apache.curator\n curator-framework\n \n \n org.apache.curator\n curator-recipes\n \n \n "}]},{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 在整合完上面的源码后，就可以直接支持配置式的定时任务了，只需要修改服务的config即可生效，但是要做到动态式的添加和删除就必须在实现一个动态的实现。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先在ElasticJobAutoConfiguration新增一个Bean"}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"/**\n * 动态任务初始化\n * @return\n */\n @Bean(initMethod = \"init\")\n @ConditionalOnMissingBean\n public DynamicJobInitialization dynamicJobInitialization() {\n return new DynamicJobInitialization(this.regCenter());\n }"}]},{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然后实现动态的类"}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"/**\n * 动态任务初始化(支持简单、流式任务)\n * @author Zzq\n * @date 2020/9/14 19:22\n */\n@Slf4j\npublic class DynamicJobInitialization extends AbstractJobInitialization {\n\n private JobStatisticsAPI jobStatisticsAPI;\n private JobSettingsAPI jobSettingsAPI;\n\n public DynamicJobInitialization(ZookeeperRegistryCenter zookeeperRegistryCenter) {\n this.jobStatisticsAPI = new JobStatisticsAPIImpl(zookeeperRegistryCenter);\n this.jobSettingsAPI = new JobSettingsAPIImpl(zookeeperRegistryCenter);\n }\n\n public void init() {\n Collection allJob = jobStatisticsAPI.getAllJobsBriefInfo();\n if (CollUtil.isNotEmpty(allJob)) {\n allJob.forEach(jobInfo -> {\n // 已下线的任务\n if (JobBriefInfo.JobStatus.CRASHED.equals(jobInfo.getStatus())) {\n try {\n Date currentDate = new Date();\n CronExpression cronExpression = new CronExpression(jobInfo.getCron());\n Date nextValidTimeAfter = cronExpression.getNextValidTimeAfter(currentDate);\n // 表达式还生效的任务\n if (ObjectUtil.isNotNull(nextValidTimeAfter)) {\n this.initJobHandler(jobInfo.getJobName());\n }\n } catch (ParseException e) {\n log.error(e.getMessage(), e);\n }\n }\n });\n }\n }\n\n /**\n * 初始化任务操作\n * @param jobName 任务名\n */\n private void initJobHandler(String jobName) {\n try {\n JobSettings jobSetting = jobSettingsAPI.getJobSettings(jobName);\n if (ObjectUtil.isNotNull(jobSetting)) {\n String jobCode = StrUtil.subBefore(jobSetting.getJobName(), StrUtil.UNDERLINE, false);\n JobClassEnum jobClassEnum = JobClassEnum.convert(jobCode);\n if (ObjectUtil.isNotNull(jobClassEnum)) {\n ElasticJobProperties.JobConfiguration configuration = new ElasticJobProperties.JobConfiguration();\n configuration.setCron(jobSetting.getCron());\n configuration.setJobParameter(jobSetting.getJobParameter());\n configuration.setShardingTotalCount(jobSetting.getShardingTotalCount());\n configuration.setDescription(jobSetting.getDescription());\n configuration.setShardingItemParameters(jobSetting.getShardingItemParameters());\n configuration.setJobClass(jobClassEnum.getClazz().getCanonicalName());\n super.initJob(jobName, JobType.valueOf(jobSetting.getJobType()), configuration);\n }\n }\n } catch (Exception e) {\n log.error(\"初始化任务操作失败: {}\", e.getMessage(), e);\n }\n }\n\n /**\n * 保存/更新任务\n * @param job\n * @param jobClass\n */\n public void addOrUpdateJob(Job job, Class extends ElasticJob> jobClass) {\n ElasticJobProperties.JobConfiguration configuration = new ElasticJobProperties.JobConfiguration();\n configuration.setCron(job.getCron());\n configuration.setJobParameter(job.getJobParameter());\n configuration.setShardingTotalCount(job.getShardingTotalCount());\n configuration.setShardingItemParameters(job.getShardingItemParameters());\n configuration.setJobClass(jobClass.getCanonicalName());\n super.initJob(job.getJobName(), JobType.valueOf(job.getJobType()), configuration);\n }\n\n @Override\n public JobTypeConfiguration getJobTypeConfiguration(String jobName, JobType jobType, JobCoreConfiguration jobCoreConfiguration) {\n String jobCode = StrUtil.subBefore(jobName, StrUtil.UNDERLINE, false);\n JobClassEnum jobClassEnum = JobClassEnum.convert(jobCode);\n if (ObjectUtil.isNotNull(jobClassEnum)) {\n if (JobType.SIMPLE.equals(jobType)) {\n return new SimpleJobConfiguration(jobCoreConfiguration, jobClassEnum.getClazz().getCanonicalName());\n } else if (JobType.DATAFLOW.equals(jobType)) {\n return new DataflowJobConfiguration(jobCoreConfiguration, jobClassEnum.getClazz().getCanonicalName(), false);\n }\n }\n return null;\n }\n}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 为什么是这样的实现？我发现每次重新发布服务后，现在的未执行的任务都会变成“已下线”，这可能跟Zookeeper有关，需要重新初始化才行，对于注解和配置式的，会自动初始化，但是动态添加的不会自动初始化。所以必须自己初始化，之前有个思路是自己建张表来维护定时，每次启动时进行初始化，但是这样太麻烦，后来实现使用elastic-job现有的API来实现，"},{"type":"text","marks":[{"type":"strong"}],"text":"即启动时，遍历Zookeeper已有的节点，然后判断Cron表达式是否过期，如果还没有过期，则重新初始化任务，初始化时配置设置了会覆盖原来的配置，所以不会有影响"},{"type":"text","text":"。然后外层可以通过MQ来新增任务，在通过服务调用去指定对应的定时逻辑即可。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"(不知道大家有没有更好的实现方案，可以初始化动态任务的)"}]},{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而配置式的，可以直接在配置文件指定并实现即可"}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"spring:\n elasticjob:\n #注册中心配置\n zookeeper:\n server-lists: 127.0.0.1:6181\n namespace: elastic-job-spring-boot-stater-demo\n #简单作业配置\n simples:\n #spring简单作业示例配置\n spring-simple-job:\n #配置简单作业，必须实现com.dangdang.ddframe.job.api.simple.SimpleJob\n job-class: com.zen.spring.boot.demo.elasticjob.job.SpringSimpleJob\n cron: 0/2 * * * * ?\n sharding-total-count: 3\n sharding-item-parameters: 0=Beijing,1=Shanghai,2=Guangzhou\n #配置监听器\n listener:\n #配置每台作业节点均执行的监听，必须实现com.dangdang.ddframe.job.lite.api.listener.ElasticJobListener\n listener-class: com.zen.spring.boot.demo.elasticjob.listener.MyElasticJobListener\n #流式作业配置\n dataflows:\n #spring简单作业示例配置\n spring-dataflow-job:\n #配置简单作业，必须实现com.dangdang.ddframe.job.api.dataflow.DataflowJob\n job-class: com.zen.spring.boot.demo.elasticjob.job.SpringDataflowJob\n cron: 0/2 * * * * ?\n sharding-total-count: 3\n sharding-item-parameters: 0=Beijing,1=Shanghai,2=Guangzhou\n streaming-process: true\n #配置监听器\n listener:\n #配置分布式场景中仅单一节点执行的监听，必须实现com.dangdang.ddframe.job.lite.api.listener.AbstractDistributeOnceElasticJobListener\n distributed-listener-class: com.zen.spring.boot.demo.elasticjob.listener.MyDistributeElasticJobListener\n started-timeout-milliseconds: 5000\n completed-timeout-milliseconds: 10000\n #脚本作业配置\n scripts:\n #脚本作业示例配置\n script-job:\n cron: 0/2 * * * * ?\n sharding-total-count: 3\n sharding-item-parameters: 0=Beijing,1=Shanghai,2=Guangzhou\n script-command-line: youPath/spring-boot-starter-demo/elastic-job-spring-boot-starter-demo/src/main/resources/script/demo.bat"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 以上整合基本可以满足现在的使用，比较期待移交Apache后的3的版本，这样可以有更多API的支持，而不用自己造轮子。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}