spring cloud stream+Kafka實現消息傳遞

(一) 爲什麼要消息傳遞在構建微服務應用程序中很重要

回答這個問題之前,首先介紹一個概念

事件驅動架構(EDA):使用異步消息實現事件之間的通信,也被稱爲消息驅動架構(MDA).

而基於EDA的方法允許開發人員構建高度解耦的系統,它可以對變更做出反應,而不需要與特定的庫或者服務緊密耦合.當與微服務結合之後,.EDA通過讓服務監聽由應用程序發出的事件流(消息)的方式,允許開發人員迅速地嚮應用程序中添加新功能.

也就是說,基於EDA的方法構建的系統具有的高度的解耦性以及擴展性;

(二) Spring Cloud Stream 簡介

1 、Spring Cloud Stream是一個由註解驅動的架構,它允許開發人員在Spring應用程序中構建消息發佈者和消費者

2、Spring Cloud可以通過Spring Cloud Stream將消息傳遞集成到Spring 的微服務中

3、Spring Cloud Stream可以使用多個消息平臺(包括Apache Kafka和RabbitMQ),而平臺的具體實現細節則被排除在應用程序之外.在應用程序中實現消息的發佈和消費是通過平臺無關的Spring接口實現的.

(三) 安裝ZooKeeper與kafka

zookeeper安裝與入門:https://blog.csdn.net/douya2016/article/details/102983218

kafka安裝與學習:https://www.cnblogs.com/qingyunzong/p/9004509.html#_label3_1

(四) Spring Cloud Stream + Kafka 實例

消息生產者生產消息,並向kafka發佈,消息消費者1從kafka中消費消息,並負責中轉;中轉之後,將其發佈到kafka,然後由comsumerGroup2組中的消息消費者消費;

項目結構如下:

其中

messgae-producer:消息生產者,負責向kafka中發佈消息;

message-comsumer1:消息消費者1,負責消息中轉

messgae-comsumer2:消息消費者2

messgae-comsumer3:消息消費者3

Spring Cloud Stream 與kafka依賴:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>

1.message-producer項目

application.yml

server:
   port: 8750
spring:
   application:
      name: producer
   cloud:
      stream:
         bindings:
            output: #通道名稱,使用stream默認的通道名稱,可以自定義
               destination: stream-demo #要寫入的消息隊列的名稱
               content-type: application/json #發送或接受什麼類型的消息
         kafka: #使用kafka作爲服務中的消息總線
            binder:
               zkNodes: localhost:2181 #zookeeper的網絡位置,如果是集羣,逗號分割
               brokers: localhost:9092 #kafka的網絡位置   
               auto-create-topics: true

啓動類

package com.dy.producer;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

/**
 * EnableBinding註解告訴Spring Cloud Steam將應用程序綁定到消息代理
 * @author dy
 *
 */
@SpringBootApplication
public class ProducerService {
	
	public static void main(String[] args) {
		SpringApplication.run(ProducerService.class, args);
	}
}

控制器MessageProducerController

package com.dy.producer.controller;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

import com.dy.producer.entiy.Message;
import com.dy.producer.service.SendService;

@RestController
public class MessageProducerController {

	@Autowired
	private SendService sendService;

	@PostMapping("/sendMsg")
	public void send(@RequestBody Message message) {
		sendService.sendMsg(message);
	}
	
}

服務類SendServiceImpl,通過默認的Source類上定義的一組通道與消息代理進行通信

package com.dy.producer.service;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.messaging.Source;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.stereotype.Service;

import com.dy.producer.entiy.Message;

@Service
@EnableBinding(Source.class)
public class SendServiceImpl implements SendService {

	@Autowired
	private Source source;

	@Override
	public void sendMsg(Message msg) {
		source.output().send(MessageBuilder.withPayload(msg).build());

	}

}

實體類Message

package com.dy.stream.entiy;

public class Message {
	String sender;
	String reciver;
	String messgaeName;
	String messageContent;
	
	public Message(String sender, String reciver, String messgaeName, String messageContent) {
		super();
		this.sender = sender;
		this.reciver = reciver;
		this.messgaeName = messgaeName;
		this.messageContent = messageContent;
	}
	
	public String getSender() {
		return sender;
	}
	public void setSender(String sender) {
		this.sender = sender;
	}
	public String getReciver() {
		return reciver;
	}
	public void setReciver(String reciver) {
		this.reciver = reciver;
	}
	public String getMessgaeName() {
		return messgaeName;
	}
	public void setMessgaeName(String messgaeName) {
		this.messgaeName = messgaeName;
	}
	public String getMessageContent() {
		return messageContent;
	}
	public void setMessageContent(String messageContent) {
		this.messageContent = messageContent;
	}
	
	@Override
	public String toString() {
		return sender + "-" + reciver + "-" + messgaeName + "-" + messageContent;
	}
}

2.message-comsumer1 消息中轉

application.yml

server:
   port: 9751
spring:
   application:
      name: comsumer1
   cloud:
      stream:
         bindings:
            input: #通道名稱,使用stream默認的通道名稱,可以自定義, 接受消息生產者生產的消息 
               destination: stream-demo #要寫入的消息隊列的名稱
 #              group: comsumerGroup1 #該屬性確保服務只處理一次
            output:
               destination: stream-demo-trans #轉發
         kafka: #使用kafka作爲服務中的消息總線
            binder:
               zkNodes: localhost:2181 #zookeeper的網絡位置,如果是集羣,逗號分割
               brokers: localhost:9092 #kafka的網絡位置
               auto-create-topics: true

啓動類

package com.dy.stream;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class ComsumerApplication1 {
	public static void main(String[] args) {
		SpringApplication.run(ComsumerApplication1.class, args);
	}
}

SendToBinder:消息代理藉口

package com.dy.stream.entiy;

import org.springframework.cloud.stream.annotation.Input;
import org.springframework.cloud.stream.annotation.Output;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.SubscribableChannel;

public interface SendToBinder {
	@Output("out")
	MessageChannel output();
	
	@Input("input")
	SubscribableChannel input();
}

TransFormService:綁定自己的SendToBinder接口，然後監聽input,返回ACK表示中轉站收到消息了，再轉發消息出去

package com.dy.stream.entiy;

import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.messaging.handler.annotation.SendTo;

/**
 * 定自己的SendToBinder接口，然後監聽input,返回ACK表示中轉站收到消息了，再轉發消息出去，代碼如下：
 * @author dy
 *
 */
@EnableBinding(SendToBinder.class)
public class TransFormService {

	@StreamListener("input")
	//SendTo註解:給別人一個反饋ACK
	@SendTo("output")
	public Object transform(Message payload) {
		System.out.println("消息中轉：" + payload.toString());
		return payload;
	}

}

3.message-comsumer2

application.yml

server:
   port: 9752
spring:
   application:
      name: comsumer2
   cloud:
      stream:
         bindings:
            input: #通道名稱,使用stream默認的通道名稱,可以自定義
               destination: stream-demo-trans #要寫入的消息隊列的名稱
#            content-type: text/plain   #發送或接受什麼類型的消息
               group: comsumerGroup2 #該屬性確保服務只處理一次
         kafka: #使用kafka作爲服務中的消息總線
            binder:
               zkNodes: localhost:2181 #zookeeper的網絡位置,如果是集羣,逗號分割
               brokers: localhost:9092 #kafka的網絡位置    
               auto-create-topics: true

啓動類

package com.dy.stream;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;

@SpringBootApplication
public class ComsumerApplication2 {	
	public static void main(String[] args) {
		SpringApplication.run(ComsumerApplication2.class, args);
	}
}

RecieveService:使用Spring Sink接口監聽默認通道input上傳入的消息:

package com.dy.stream.entiy;

import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;

@EnableBinding(Sink.class)
public class RecieveService {
	
	// 每次收到來自input通道消息時, Spring Cloud Stream將執行此方法
	@StreamListener(Sink.INPUT)
	public void recieve(Message payload) {
		System.out.println(payload.toString());
	}
}