flume java.lang.IllegalArgumentException: Timestamp must be positive

報錯:

java.lang.IllegalArgumentException: Timestamp must be positive
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownField(TimestampRoundDownUtil.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownTimeStampMinutes(TimestampRoundDownUtil.java:58)
        at org.apache.flume.formatter.output.BucketPath.roundDown(BucketPath.java:346)
        at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:242)
        at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:432)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:380)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)
02 Mar 2020 14:42:19,819 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:160)  - Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: java.lang.IllegalArgumentException: Timestamp must be positive
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:463)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Timestamp must be positive
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownField(TimestampRoundDownUtil.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownTimeStampMinutes(TimestampRoundDownUtil.java:58)
        at org.apache.flume.formatter.output.BucketPath.roundDown(BucketPath.java:346)
        at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:242)
        at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:432)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:380)
        ... 3 more
02 Mar 2020 14:42:24,821 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.hdfs.HDFSEventSink.process:459)  - process failed
java.lang.IllegalArgumentException: Timestamp must be positive
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownField(TimestampRoundDownUtil.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownTimeStampMinutes(TimestampRoundDownUtil.java:58)
        at org.apache.flume.formatter.output.BucketPath.roundDown(BucketPath.java:346)
        at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:242)
        at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:432)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:380)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)

原因:

       kafka生產者沒有提供 時間戳。

解決:

一、修改flume配置文件

添加:k2h.sinks.k1.hdfs.useLocalTimeStamp=true

二、kafka 生產者添加時間戳

new ProducerRecord<String, String>(topic,null,System.currentTimeMillis(), UUID.randomUUID().toString(), messageStr)

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章