Kafka 集成SpringBoot

1.環境準備

1.Kafka集羣環境準備

1.準備一個Kafka集羣環境並啓動

Kafka 3.6.1 集羣安裝與部署

2.創建first Topic

/usr/kafka/kafka_2.13-3.6.1/bin/kafka-topics.sh --bootstrap-server 192.168.58.130:9092 --create --partitions 1 --replication-factor 3 --topic first

2.SpringBoot環境準備

通過Spring Initializr新建SpringBoot項目

初始化後的POM文件如下：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>3.2.3</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>cn.coreqi</groupId>
	<artifactId>springboot_kafka</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>springboot_kafka</name>
	<description>springboot_kafka</description>
	<properties>
		<java.version>21</java.version>
	</properties>
	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.kafka</groupId>
			<artifactId>spring-kafka</artifactId>
		</dependency>

		<dependency>
			<groupId>org.projectlombok</groupId>
			<artifactId>lombok</artifactId>
			<optional>true</optional>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.springframework.kafka</groupId>
			<artifactId>spring-kafka-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
				<configuration>
					<excludes>
						<exclude>
							<groupId>org.projectlombok</groupId>
							<artifactId>lombok</artifactId>
						</exclude>
					</excludes>
				</configuration>
			</plugin>
		</plugins>
	</build>

</project>

2.SpringBoot 生產者

1.修改 SpringBoot 配置文件 application.propeties, 添加生產者相關信息

# 應用名稱
spring.application.name=springboot_kafka

# 指定 kafka 的地址
spring.kafka.bootstrap-servers=192.168.58.130:9092,192.168.58.131:9092,192.168.58.132:9092

#指定 key 和 value 的序列化器
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

2.創建 controller 從瀏覽器接收數據, 並寫入指定的 topic

package cn.coreqi.springboot.controller;

import jakarta.annotation.Resource;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class ProducerController {
    // Kafka 模板用來向 kafka 發送數據
    @Resource
    KafkaTemplate<String, String> kafka;

    @RequestMapping("/coreqi")
    public String data(String msg) {
        kafka.send("first", msg);
        return "ok";
    }
}

3.啓動 Kafka 消費者

/usr/kafka/kafka_2.13-3.6.1/bin/kafka-console-consumer.sh --bootstrap-server 192.168.58.130:9092 --topic first

4.在瀏覽器中訪問/coreqi 接口並向其發送數據，觀察 kafka 消費者控制檯情況

http://localhost:8080/coreqi?msg=hello

3.SpringBoot 消費者

1.修改 SpringBoot 配置文件 application.propeties, 添加消費者相關信息

# 應用名稱
spring.application.name=springboot_kafka

# 指定 kafka 的地址
spring.kafka.bootstrap-servers=192.168.58.130:9092,192.168.58.131:9092,192.168.58.132:9092

# 指定 key 和 value 的反序列化器
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer

#指定消費者組的 group_id
spring.kafka.consumer.group-id=coreqi

2.創建類消費 Kafka 中指定 topic 的數據

package cn.coreqi.springboot.config;

import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.annotation.KafkaListener;

@Configuration
public class KafkaConsumer {
    // 指定要監聽的 topic
    @KafkaListener(topics = "first")
    public void consumeTopic(String msg) { // 參數: 收到的 value
        System.out.println("收到的信息: " + msg);
    }
}

3.啓動 kafka 生產者

/usr/kafka/kafka_2.13-3.6.1/bin/kafka-console-producer.sh --bootstrap-server 192.168.58.130:9092 --topic first

4.啓動應用 觀察 IDEA 控制檯數據打印