dolphinscheduler版本2.0.0
測試步驟:
1.新建數據源測試成功
2.新建工作流執行失敗日誌(從日誌看沒有看錯什麼異常)
[LOG-PATH]: /var/log/udp/1.0.0.0/dolphinscheduler/8136281159680_2/2773/4914.log, [HOST]: 10.30.10.62
2023-03-21 16:20:38.046 [INFO] - [taskAppId=TASK-8136281159680_2-2773-4914]:[119] - Full sql parameters: SqlParameters{type='CLICKHOUSE', datasource=7, sql='select * from ddp_fat_dwh.temp limit 10
', sqlType=0, sendEmail=false, displayRows=10, limit=0, udfs='', showType='null', connParams='', groupId='0', title='', preStatements=[], postStatements=[]}
2023-03-21 16:20:38.052 [INFO] - [taskAppId=TASK-8136281159680_2-2773-4914]:[120] - sql type : CLICKHOUSE, datasource : 7, sql : select * from ddp_fat_dwh.temp limit 10
, localParams : [],udfs : ,showType : null,connParams : ,varPool : [] ,query max result limit 0
2023-03-21 16:20:38.053 [INFO] - [taskAppId=TASK-8136281159680_2-2773-4914]:[443] - after replace sql , preparing : select * from ddp_fat_dwh.temp limit 10
2023-03-21 16:20:38.053 [INFO] - [taskAppId=TASK-8136281159680_2-2773-4914]:[452] - Sql Params are replaced sql , parameters:
2023-03-21 16:20:38.053 [INFO] - [taskAppId=TASK-8136281159680_2-2773-4914]:[522] - can't find udf function resource
3.去對應的work節點查看日誌,具體日誌路徑以爲work的log配置文件爲準:/var/log/udp/1.0.0.0/dolphinscheduler
2023-03-21 16:52:22.958 [INFO] - [taskAppId=TASK-8136281159680_2-2771-4915]:[443] - after replace sql , preparing : SELECT 1;
2023-03-21 16:52:22.958 [INFO] - [taskAppId=TASK-8136281159680_2-2771-4915]:[452] - Sql Params are replaced sql , parameters:
2023-03-21 16:52:22.959 [INFO] - [taskAppId=TASK-8136281159680_2-2771-4915]:[522] - can't find udf function resource
2023-03-21 16:52:22.960 [ERROR] org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[210] - task scheduler failure
java.lang.NoClassDefFoundError: org/apache/http/conn/ConnectionKeepAliveStrategy
at ru.yandex.clickhouse.ClickHouseConnectionImpl.<init>(ClickHouseConnectionImpl.java:73)
at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:55)
at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:47)
at ru.yandex.clickhouse.ClickHouseDriver.connect(ClickHouseDriver.java:29)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.dolphinscheduler.plugin.task.datasource.clickhouse.ClickHouseDatasourceProcessor.getConnection(ClickHouseDatasourceProcessor.java:102)
at org.apache.dolphinscheduler.plugin.task.datasource.DatasourceUtil.getConnection(DatasourceUtil.java:59)
at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.executeFuncAndSql(SqlTask.java:184)
at org.apache.dolphinscheduler.plugin.task.sql.SqlTask.handle(SqlTask.java:155)
at org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread.run(TaskExecuteThread.java:192)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.http.conn.ConnectionKeepAliveStrategy
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at org.apache.dolphinscheduler.spi.plugin.DolphinPluginClassLoader.loadClass(DolphinPluginClassLoader.java:81)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 16 common frames omitted
看到具體的異常日誌大概知道原因了,需要將相關的依賴上傳到work節點對應的lib目錄/srv/udp/1.0.0.0/dolphinscheduler/lib/plugin/task/sql
4.clickhouse的依賴很多將所有的依賴打包輸出
4.1新建一個maven項目添加clickhouse依賴
4.2執行命令mvn dependency:copy-dependencies -DoutputDirectory=C:\\my\\out
4.3.將依賴上傳並且授權
5.重啓work節點搞定!!!