Kafka新舊版本命令問題

1、異常彙總

輸入如下命令生成一個生產者/usr/local/chen/kafka/bin/kafka-console-producer.sh --broker-list 192.168.43.175:2181,192.168.43.176:2181,192.168.43.177:2181 --topic chentest7。出現以下異常。

>[2020-04-12 15:27:51,725] WARN [Producer clientId=console-producer] Bootstrap broker 192.168.43.175:2181 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2020-04-12 15:27:51,830] WARN [Producer clientId=console-producer] Bootstrap broker 192.168.43.177:2181 (id: -3 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2020-04-12 15:27:51,932] WARN [Producer clientId=console-producer] Bootstrap broker 192.168.43.177:2181 (id: -3 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2020-04-12 15:27:52,036] WARN [Producer clientId=console-producer] Bootstrap broker 192.168.43.175:2181 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2020-04-12 15:27:52,143] WARN [Producer clientId=console-producer] Bootstrap broker 192.168.43.176:2181 (id: -2 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2020-04-12 15:27:52,245] WARN [Producer clientId=console-producer] Bootstrap broker 192.168.43.176:2181 (id: -2 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)

2、問題分析

  這是由於新舊版本kafka命令不同。

3、問題解決

  使用新版本的kafka啓動命令即可。/usr/local/chen/kafka/bin/kafka-console-producer.sh --broker-list 192.168.43.175:9092,192.168.43.176:9092,192.168.43.177:9092 --topic chentest6用來啓動生產者。輸入信息即可被消費者監聽到。

然後使用/usr/local/chen/kafka/bin/kafka-console-consumer.sh --bootstrap-server 192.168.43.175:9092,192.168.43.176:9092,192.168.43.177:9092 --topic chentest6 --from-beginning命令生成消費者。我們發現消費者已經成功的消費到信息。

4、使用java生產數據

簡單的定義一個類，配置kafka的相關信息，用來生成數據。

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;

import java.util.Properties;

/**
 * @author Chen
 * @version 1.0
 * @date 2020/4/10 8:03
 * @description:
 */
public class ProducerDemo {
    private final KafkaProducer<String, String> producer;

//    public final static String TOPIC = "access-log";
    public final static String TOPIC = "chentest6";

    private ProducerDemo() {
        Properties props = new Properties();
        props.put("zookeeper.connect", "192.168.43.175:2181,192.168.43.176:2181,192.168.43.177:2181");
        props.put("acks", "all");//所有follower都響應了才認爲消息提交成功，即"committed"
        props.put("retries", 0);//retries = MAX 無限重試，直到你意識到出現了問題:)
        props.put("batch.size", 16384);//producer將試圖批處理消息記錄，以減少請求次數.默認的批量處理消息字節數
        //batch.size當批量的數據大小達到設定值後，就會立即發送，不顧下面的linger.ms
        props.put("linger.ms", 1);//延遲1ms發送，這項設置將通過增加小的延遲來完成--即，不是立即發送一條記錄，producer將會等待給定的延遲時間以允許其他消息記錄發送，這些消息記錄可以批量處理
        props.put("buffer.memory", 33554432);//producer可以用來緩存數據的內存大小。
        props.put("key.serializer",
                "org.apache.kafka.common.serialization.IntegerSerializer");
        props.put("value.serializer",
                "org.apache.kafka.common.serialization.StringSerializer");

        producer = new KafkaProducer<String, String>(props);
    }

    public void produce() {
        int messageNo = 1;
        final int COUNT = 5;

        while (messageNo < COUNT) {
            String key = String.valueOf(messageNo);
            String data = String.format("hello KafkaProducer message %s from hubo chen ", key);

            try {
                producer.send(new ProducerRecord<String, String>(TOPIC, data));
            } catch (Exception e) {
                e.printStackTrace();
            }

            messageNo++;
        }

        producer.close();
    }

    public static void main(String[] args) {
        ProducerDemo producerDemo = new ProducerDemo();
        producerDemo.produce();
    }

}

運行後不出意外出現了異常。

Exception in thread "main" org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.
	at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:463)
	at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:453)
	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:62)
	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:75)
	at org.apache.kafka.clients.producer.ProducerConfig.<init>(ProducerConfig.java:359)
	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:287)
	at ProducerDemo.<init>(ProducerDemo.java:35)
	at ProducerDemo.main(ProducerDemo.java:59)

同樣是版本問題，在新版本中，使用props.put("bootstrap.servers", "192.168.43.175:9092,192.168.43.176:9092,192.168.43.177:9092");配置服務器。修改代碼如下：

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;

import java.util.Properties;

/**
 * @author Chen
 * @version 1.0
 * @date 2020/4/10 8:03
 * @description:
 */
public class ProducerDemo {
    private final KafkaProducer<String, String> producer;

//    public final static String TOPIC = "access-log";
    public final static String TOPIC = "chentest6";

    private ProducerDemo() {
        Properties props = new Properties();
        props.put("bootstrap.servers", "192.168.43.175:9092,192.168.43.176:9092,192.168.43.177:9092");//xxx服務器ip
//        props.put("bootstrap.servers", "192.168.43.120:9092");//xxx服務器ip
//        props.put("bootstrap.servers", "192.168.43.175:9092");//xxx服務器ip
        props.put("acks", "all");//所有follower都響應了才認爲消息提交成功，即"committed"
        props.put("retries", 0);//retries = MAX 無限重試，直到你意識到出現了問題:)
        props.put("batch.size", 16384);//producer將試圖批處理消息記錄，以減少請求次數.默認的批量處理消息字節數
        //batch.size當批量的數據大小達到設定值後，就會立即發送，不顧下面的linger.ms
        props.put("linger.ms", 1);//延遲1ms發送，這項設置將通過增加小的延遲來完成--即，不是立即發送一條記錄，producer將會等待給定的延遲時間以允許其他消息記錄發送，這些消息記錄可以批量處理
        props.put("buffer.memory", 33554432);//producer可以用來緩存數據的內存大小。
        props.put("key.serializer",
                "org.apache.kafka.common.serialization.IntegerSerializer");
        props.put("value.serializer",
                "org.apache.kafka.common.serialization.StringSerializer");

        producer = new KafkaProducer<String, String>(props);
    }

    public void produce() {
        int messageNo = 1;
        final int COUNT = 5;

        while (messageNo < COUNT) {
            String key = String.valueOf(messageNo);
            String data = String.format("hello KafkaProducer message %s from hubo chen ", key);

            try {
                producer.send(new ProducerRecord<String, String>(TOPIC, data));
            } catch (Exception e) {
                e.printStackTrace();
            }

            messageNo++;
        }

        producer.close();
    }

    public static void main(String[] args) {
        ProducerDemo producerDemo = new ProducerDemo();
        producerDemo.produce();
    }

}

直接運行，查看服務器消費者信息。

可以成功消費到數據。

4、總結

  軟件的更新修改了BUG，或許也添加了新的功能，但是對使用者來說有點不太友好，要是自動檢測到某個命令是舊命令，然後提示用戶輸入一個新的命令，是不是想想都激動。算了當前幻想，好好看官方文檔吧！

點個贊再走唄！歡迎留言哦！