Spring Cloud Stream消費失敗後的處理策略（四）：使用DLQ隊列（Kafka版）

Spring Cloud Stream消費失敗後的處理策略（四）：使用DLQ隊列（Kafka版）

之前寫了一個一個DLQ的rabbit版本，接下來說一下kafka版本的。因爲kafka的管理監控界面沒有那麼詳細（我太菜，沒找到）。

動手試一下：

準備一個會消費失敗的例子，可以直接沿用前文的工程。也可以新建一個，然後創建如下代碼的邏輯：

public class TestController {
    @Autowired
    private TestTopic testTopic;
    @RequestMapping("entity")
    public void TestEntity(@RequestParam String message){
        Message message1 = new Message("1", message, new Date()); testTopic.output1().send(MessageBuilder.withPayload(message1).build());
    }
}

@Data
@NoArgsConstructor
@AllArgsConstructor
public class Message implements Serializable {
    String id;
    String msg;
    Date date;
}

public interface TestTopic {
    String OUTPUT1 = "example-topic-output1";
    String INPUT1 = "example-topic-input1";
    String ERRORINPUT = "errorChannelInput";
    String ERROROUTPUT = "errorChannelOutput";

    @Output(OUTPUT1)
    MessageChannel output1();

    @Input(INPUT1)
    SubscribableChannel input1();

    @Input(ERRORINPUT)
    SubscribableChannel errorInput();

    @Output(ERROROUTPUT)
    MessageChannel errorOutput();
}

@Slf4j
@Component
@EnableBinding(TestTopic.class)
public class TestListener {
    @StreamListener(TestTopic.INPUT1)
    public void receiveEntity(Message message) {
        log.info("Received payload : {}", message);
        throw new RuntimeException("Message consumer failed!");
    }
}

消息消費的時候主動拋出了一個異常來模擬消息的消費失敗。

在啓動應用之前，還要記得配置一下輸入輸出通道對應的物理目標（topic名）、並設置一下分組，比如：

# 爲消費者設置組，防止重複消費
spring.cloud.stream.bindings.example-topic-input1.group=Service-A
spring.cloud.stream.bindings.errorChannel.group=Service-A
# 爲example-topic-input通道綁定topic
spring.cloud.stream.bindings.example-topic-input1.destination=greeting1
spring.cloud.stream.bindings.errorChannelInput.destination=errorTopic
# example-topic-input通道發送消息失敗時的重試次數
spring.cloud.stream.bindings.example-topic-input1.consumer.max-attempts=1
# 輸出example-topic-output通道綁定的topic
spring.cloud.stream.bindings.example-topic-output1.destination=greeting1
spring.cloud.stream.bindings.errorChannelOutput.destination=errorTopic
# kafka服務器地址
spring.cloud.stream.kafka.binder.brokers=ip:port
server.port=8080
# 是否綁定死信隊列,會立即轉發至死信隊列，若失敗則會發送3次
spring.cloud.stream.kafka.bindings.example-topic-input1.consumer.enable-dlq=true
spring.cloud.stream.kafka.bindings.example-topic-input1.consumer.dlq-name=errorTopic

這裏加入了兩個重要配置

spring.cloud.stream.kafka.bindings.example-topic-input1.consumer.enable-dlq=true，

spring.cloud.stream.kafka.bindings.example-topic-input1.consumer.dlq-name=errorTopic

用來開啓DLQ（死信隊列）並指明死信隊列名。完成了上面配置之後，啓動應用並訪問`localhost:8080/entity?message=aaa

最後打印日誌：

2019-09-18 16:37:13.129  INFO 9184 --- [container-0-C-1] c.c.f.stream.listener.TestListener       : Received payload : Message(id=1, msg=aaaAASAsa, date=Wed Sep 18 16:37:13 CST 2019)

org.springframework.messaging.MessagingException: Exception thrown while invoking com.chinatower.framework.stream.listener.TestListener#receiveEntity[1 args]; nested exception is java.lang.RuntimeException: Message consumer failed!
	at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:63) ~[spring-cloud-stream-2.0.1.RELEASE.jar:2.0.1.RELEASE]

Caused by: java.lang.RuntimeException: Message consumer failed!
	at com.chinatower.framework.stream.listener.TestListener.receiveEntity(TestListener.java:26) ~[classes/:na]


2019-09-18 16:37:13.131 ERROR 9184 --- [container-0-C-1] o.s.integration.handler.LoggingHandler   : org.springframework.messaging.MessagingException: Exception thrown while invoking com.chinatower.framework.stream.listener.TestListener#receiveEntity[1 args]; nested exception is java.lang.RuntimeException: Message consumer failed!, failedMessage=GenericMessage [payload=byte[66], headers={kafka_offset=39, scst_nativeHeadersPresent=true, kafka_consumer=org.apache.kafka.clients.consumer.KafkaConsumer@3a3d1789, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, contentType=application/json, kafka_receivedTopic=greeting1, kafka_receivedTimestamp=1568795833086}]
	at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:63)

Caused by: java.lang.RuntimeException: Message consumer failed!
	at com.chinatower.framework.stream.listener.TestListener.receiveEntity(TestListener.java:26)


2019-09-18 16:37:13.173  INFO 9184 --- [container-0-C-1] c.c.f.stream.listener.TestListener       : errorReceive payload : {"id":"1","msg":"aaaAASAsa","date":"2019-09-18T08:37:13.084+0000"}

如此一來，就可以做到自己接受死信隊列裏的消息，做預防措施。