整合Elastic-Job(支持動態任務)

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 最近公司的項目需要用到分佈式任務調度，在結合多款開源框架後決定使用噹噹的Elastic-job。不知道大家有沒有這樣的需求，就是動態任務。之前比較了xxl-job和elastic-job發現，都只是支持註解或者配置以及後臺添加現有的任務，不支持動態添加。比如：類似訂單半小時後自動取消的場景。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" xxl-job理論上來說是可以支持的，但是需要高度整合admin端的程序，然後開放對應的接口才可以給其他服務調用，這樣本質直接改源碼對後期的升級十分不便，最後放棄了xxl-job。elastic-job在移交Apache後的版本規劃中，有提到API的開放，但是目前還沒有穩定版，所以只能使用之前的2.1.5的版本來做。在Github搜了很多整合方案，最後決定選擇下面的來實現。"}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"\n com.github.xjzrc.spring.boot\n elastic-job-lite-spring-boot-starter\n ${lasted.release.version}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5c/5c71947a0cced6a7664f6dfe6832e8de.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 因爲要做的是動態的，所以這裏沒有直接使用maven座標引入，直接將源碼全部接入項目來使用，這樣比較靈活，因爲底層本質上還是用elastic-job的東西。下面引入elastic-job座標"}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":" \n \t\tcom.dangdang\n \t\telastic-job-lite-spring\n \t\t${elastic-job.version}\n \n \n \t\tcom.dangdang\n \t\telastic-job-lite-lifecycle\n \t\t${elastic-job.version}\n \n \n org.eclipse.jetty.aggregate\n jetty-all-server\n \n \n org.apache.curator\n curator-framework\n \n \n org.apache.curator\n curator-recipes\n \n \n "}]},{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 在整合完上面的源碼後，就可以直接支持配置式的定時任務了，只需要修改服務的config即可生效，但是要做到動態式的添加和刪除就必須在實現一個動態的實現。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先在ElasticJobAutoConfiguration新增一個Bean"}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"/**\n * 動態任務初始化\n * @return\n */\n @Bean(initMethod = \"init\")\n @ConditionalOnMissingBean\n public DynamicJobInitialization dynamicJobInitialization() {\n return new DynamicJobInitialization(this.regCenter());\n }"}]},{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然後實現動態的類"}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"/**\n * 動態任務初始化(支持簡單、流式任務)\n * @author Zzq\n * @date 2020/9/14 19:22\n */\n@Slf4j\npublic class DynamicJobInitialization extends AbstractJobInitialization {\n\n private JobStatisticsAPI jobStatisticsAPI;\n private JobSettingsAPI jobSettingsAPI;\n\n public DynamicJobInitialization(ZookeeperRegistryCenter zookeeperRegistryCenter) {\n this.jobStatisticsAPI = new JobStatisticsAPIImpl(zookeeperRegistryCenter);\n this.jobSettingsAPI = new JobSettingsAPIImpl(zookeeperRegistryCenter);\n }\n\n public void init() {\n Collection allJob = jobStatisticsAPI.getAllJobsBriefInfo();\n if (CollUtil.isNotEmpty(allJob)) {\n allJob.forEach(jobInfo -> {\n // 已下線的任務\n if (JobBriefInfo.JobStatus.CRASHED.equals(jobInfo.getStatus())) {\n try {\n Date currentDate = new Date();\n CronExpression cronExpression = new CronExpression(jobInfo.getCron());\n Date nextValidTimeAfter = cronExpression.getNextValidTimeAfter(currentDate);\n // 表達式還生效的任務\n if (ObjectUtil.isNotNull(nextValidTimeAfter)) {\n this.initJobHandler(jobInfo.getJobName());\n }\n } catch (ParseException e) {\n log.error(e.getMessage(), e);\n }\n }\n });\n }\n }\n\n /**\n * 初始化任務操作\n * @param jobName 任務名\n */\n private void initJobHandler(String jobName) {\n try {\n JobSettings jobSetting = jobSettingsAPI.getJobSettings(jobName);\n if (ObjectUtil.isNotNull(jobSetting)) {\n String jobCode = StrUtil.subBefore(jobSetting.getJobName(), StrUtil.UNDERLINE, false);\n JobClassEnum jobClassEnum = JobClassEnum.convert(jobCode);\n if (ObjectUtil.isNotNull(jobClassEnum)) {\n ElasticJobProperties.JobConfiguration configuration = new ElasticJobProperties.JobConfiguration();\n configuration.setCron(jobSetting.getCron());\n configuration.setJobParameter(jobSetting.getJobParameter());\n configuration.setShardingTotalCount(jobSetting.getShardingTotalCount());\n configuration.setDescription(jobSetting.getDescription());\n configuration.setShardingItemParameters(jobSetting.getShardingItemParameters());\n configuration.setJobClass(jobClassEnum.getClazz().getCanonicalName());\n super.initJob(jobName, JobType.valueOf(jobSetting.getJobType()), configuration);\n }\n }\n } catch (Exception e) {\n log.error(\"初始化任務操作失敗: {}\", e.getMessage(), e);\n }\n }\n\n /**\n * 保存/更新任務\n * @param job\n * @param jobClass\n */\n public void addOrUpdateJob(Job job, Class extends ElasticJob> jobClass) {\n ElasticJobProperties.JobConfiguration configuration = new ElasticJobProperties.JobConfiguration();\n configuration.setCron(job.getCron());\n configuration.setJobParameter(job.getJobParameter());\n configuration.setShardingTotalCount(job.getShardingTotalCount());\n configuration.setShardingItemParameters(job.getShardingItemParameters());\n configuration.setJobClass(jobClass.getCanonicalName());\n super.initJob(job.getJobName(), JobType.valueOf(job.getJobType()), configuration);\n }\n\n @Override\n public JobTypeConfiguration getJobTypeConfiguration(String jobName, JobType jobType, JobCoreConfiguration jobCoreConfiguration) {\n String jobCode = StrUtil.subBefore(jobName, StrUtil.UNDERLINE, false);\n JobClassEnum jobClassEnum = JobClassEnum.convert(jobCode);\n if (ObjectUtil.isNotNull(jobClassEnum)) {\n if (JobType.SIMPLE.equals(jobType)) {\n return new SimpleJobConfiguration(jobCoreConfiguration, jobClassEnum.getClazz().getCanonicalName());\n } else if (JobType.DATAFLOW.equals(jobType)) {\n return new DataflowJobConfiguration(jobCoreConfiguration, jobClassEnum.getClazz().getCanonicalName(), false);\n }\n }\n return null;\n }\n}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 爲什麼是這樣的實現？我發現每次重新發布服務後，現在的未執行的任務都會變成“已下線”，這可能跟Zookeeper有關，需要重新初始化纔行，對於註解和配置式的，會自動初始化，但是動態添加的不會自動初始化。所以必須自己初始化，之前有個思路是自己建張表來維護定時，每次啓動時進行初始化，但是這樣太麻煩，後來實現使用elastic-job現有的API來實現，"},{"type":"text","marks":[{"type":"strong"}],"text":"即啓動時，遍歷Zookeeper已有的節點，然後判斷Cron表達式是否過期，如果還沒有過期，則重新初始化任務，初始化時配置設置了會覆蓋原來的配置，所以不會有影響"},{"type":"text","text":"。然後外層可以通過MQ來新增任務，在通過服務調用去指定對應的定時邏輯即可。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"(不知道大家有沒有更好的實現方案，可以初始化動態任務的)"}]},{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而配置式的，可以直接在配置文件指定並實現即可"}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"spring:\n elasticjob:\n #註冊中心配置\n zookeeper:\n server-lists: 127.0.0.1:6181\n namespace: elastic-job-spring-boot-stater-demo\n #簡單作業配置\n simples:\n #spring簡單作業示例配置\n spring-simple-job:\n #配置簡單作業，必須實現com.dangdang.ddframe.job.api.simple.SimpleJob\n job-class: com.zen.spring.boot.demo.elasticjob.job.SpringSimpleJob\n cron: 0/2 * * * * ?\n sharding-total-count: 3\n sharding-item-parameters: 0=Beijing,1=Shanghai,2=Guangzhou\n #配置監聽器\n listener:\n #配置每臺作業節點均執行的監聽，必須實現com.dangdang.ddframe.job.lite.api.listener.ElasticJobListener\n listener-class: com.zen.spring.boot.demo.elasticjob.listener.MyElasticJobListener\n #流式作業配置\n dataflows:\n #spring簡單作業示例配置\n spring-dataflow-job:\n #配置簡單作業，必須實現com.dangdang.ddframe.job.api.dataflow.DataflowJob\n job-class: com.zen.spring.boot.demo.elasticjob.job.SpringDataflowJob\n cron: 0/2 * * * * ?\n sharding-total-count: 3\n sharding-item-parameters: 0=Beijing,1=Shanghai,2=Guangzhou\n streaming-process: true\n #配置監聽器\n listener:\n #配置分佈式場景中僅單一節點執行的監聽，必須實現com.dangdang.ddframe.job.lite.api.listener.AbstractDistributeOnceElasticJobListener\n distributed-listener-class: com.zen.spring.boot.demo.elasticjob.listener.MyDistributeElasticJobListener\n started-timeout-milliseconds: 5000\n completed-timeout-milliseconds: 10000\n #腳本作業配置\n scripts:\n #腳本作業示例配置\n script-job:\n cron: 0/2 * * * * ?\n sharding-total-count: 3\n sharding-item-parameters: 0=Beijing,1=Shanghai,2=Guangzhou\n script-command-line: youPath/spring-boot-starter-demo/elastic-job-spring-boot-starter-demo/src/main/resources/script/demo.bat"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 以上整合基本可以滿足現在的使用，比較期待移交Apache後的3的版本，這樣可以有更多API的支持，而不用自己造輪子。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}