ES&Logstash報錯備案:使用logstash-input-jdbc導入數據到ES中出的錯誤

錯誤1:原因,配置文件格式問題,可能是少了空格之類的。導致配置文件無法正確讀取。

檢查配置文件的空格,以及驅動庫的位置、sql文件位置路徑是否正確。

 

錯誤2:報錯。但是不影響數據導入。可以忽略。在此Mark一下。

logstash報錯如下。

09:16:33.882 [[main]>worker6] ERROR logstash.outputs.elasticsearch - Retrying individual actions
09:16:33.882 [[main]>worker2] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@333b75e7 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.882 [[main]>worker5] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@3cb3eeba on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.902 [[main]>worker6] ERROR logstash.outputs.elasticsearch - Action
09:16:33.902 [[main]>worker3] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@1ae963b6 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.912 [[main]>worker6] ERROR logstash.outputs.elasticsearch - Action
09:16:33.892 [[main]>worker4] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@4f3848a on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.912 [[main]>worker6] ERROR logstash.outputs.elasticsearch - Action
09:16:33.912 [[main]>worker5] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@6ff3fb37 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.902 [[main]>worker2] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@333b75e7 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})
09:16:33.922 [[main]>worker3] INFO  logstash.outputs.elasticsearch - retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7@d5b137a on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@9d14abd[Running, pool size = 8, active threads = 8, queued tasks = 50, completed tasks = 2726]]"})

ES報錯如下

[2019-11-12T09:21:29,490][DEBUG][o.e.a.b.TransportShardBulkAction] [node-1] [gcrz3][3] failed to execute bulk item (index) index {[gcrz3][EEA_S_USER_ROLE][109876], source[{"@timestamp":"2019-11-12T01:11:00.485Z","user_id":"702996","role_id":"440001","@version":"1","id":"109876","type":"EEA_S_USER_ROLE"}]}
org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: failed to process cluster event (put-mapping) within 30s
        at org.elasticsearch.cluster.service.ClusterService.lambda$onTimeout$5(ClusterService.java:497) ~[elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:527) ~[elasticsearch-5.2.0.jar:5.2.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_73]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_73]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]
[2019-11-12T09:21:29,490][DEBUG][o.e.a.b.TransportShardBulkAction] [node-1] [gcrz3][2] failed to execute bulk item (index) index {[gcrz3][EEA_S_USER_ROLE][917566], source[{"@timestamp":"2019-11-12T01:11:00.482Z","user_id":"968327","role_id":"440001","@version":"1","id":"917566","type":"EEA_S_USER_ROLE"}]}
org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: failed to process cluster event (put-mapping) within 30s
        at org.elasticsearch.cluster.service.ClusterService.lambda$onTimeout$5(ClusterService.java:497) ~[elasticsearch-5.2.0.jar:5.2.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:527) ~[elasticsearch-5.2.0.jar:5.2.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_73]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_73]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]

初步判斷是jdk和導入Oracle的ojdbc版本不對應的問題。顯示時debug調試模式。暫時不影響數據的導入。就是logstash報錯看着有點難受。不過ES要求jdk8以上,Oracle11的要求ojdbc6,應該是jdk版本對應的問題。

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章