zookeeper
下載並解壓
wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz
tar -zxvf ./zookeeper-3.4.14.tar.gz
修改配置文件
cd ./zookeeper-3.4.14/conf/
cp zoo_sample.cfg zoo.cfg
vi zoo.cfg
#下面這幾個是默認就有的,我用的是默認的,我沒有配置額外的東西,你們可以根據實際情況配置像保存路徑啊什麼的
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper
clientPort=2181
配置環境變量
vim /etc/profile
#zk
export ZK_HOME=/usr/local/bigdata/zookeeper-3.4.14
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$SCALA_HOME/bin:$SPARK_HOME/bin:$ZK_HOME/bin:$KAFKA_HOME/bin
source /etc/profile
啓動zk
zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/bigdata/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
查看狀態
zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/bigdata/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: standalone
kafka
解壓
tar -zxvf kafka_2.11-0.11.0.2.tgz
修改配置
- 下面的配置我用主機名(我事先配置好ip映射),發現命令行生產者消費者正常,但是java api生產者消費者連不上,必須用ip纔行,目前沒有查詢到原因
cd ./kafka_2.11-0.11.0.2/config/
vi server.properties
listeners=PLAINTEXT://192.168.199.100:9092
advertised.listeners=PLAINTEXT://192.168.199.100:9092
zookeeper.connect=192.168.199.100:2181
host.name=192.168.199.100
打開下面這個註釋,要不然你kafka刪除只是標記刪除
修改我們這個broker保存的分區數,kafka單機版只能讓這個單機多放一些分區了
其他的默認的就可以
配置環境變量
```bash
vim /etc/profile
#kafka
export KAFKA_HOME=/usr/local/bigdata/kafka_2.11-0.11.0.2
PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$SCALA_HOME/bin:$SPARK_HOME/bin:$ZK_HOME/bin:$KAFKA_HOME/bin
source /etc/profile
啓動kafka
kafka-server-start.sh /usr/local/bigdata/kafka_2.11-0.11.0.2/config/server.properties &
會出現一堆日誌,太多了,就不截圖了
新建啓動腳本
發現這個命令太麻煩了
cd /usr/local/bigdata/kafka_2.11-0.11.0.2/bin
vi start-kafka-server.sh
#!/bin/sh
/usr/local/bigdata/kafka_2.11-0.11.0.2/bin/kafka-server-start.sh /usr/local/bigdata/kafka_2.11-0.11.0.2/config/server.properties &
vi stop-kafka-server.sh
#!/bin/sh
/usr/local/bigdata/kafka_2.11-0.11.0.2/bin/kafka-server-stop.sh /usr/local/bigdata/kafka_2.11-0.11.0.2/config/server.properties &
chmod 777 start-kafka-server.sh
chmod 777 stop-kafka-server.sh
創建一個topic
kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
Created topic "test".
發送接收消息
用命令起一個消費者監聽消息
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
起一個生產者發送消息
kafka-console-producer.sh --broker-list localhost:9092 --topic test
輸入helloowrd
可以看到消費者接收到了
java api連不上
vi server.properties
看一下這4個配置,只能用ip,用主機名不行
listeners=PLAINTEXT://192.168.199.100:9092
advertised.listeners=PLAINTEXT://192.168.199.100:9092
zookeeper.connect=192.168.199.100:2181
host.name=192.168.199.100
然後我就解決了,還沒解決可能是防火牆和端口
防火牆(root)
systemctl stop firewalld.service #停止firewall
systemctl disable firewalld.service #禁止firewall開機啓動
systemctl status firewalld.service #查看firewall的狀態
打開相關端口(root)
firewall-cmd --zone=public --add-port=2181/tcp --permanent
firewall-cmd --zone=public --add-port=9092/tcp --permanent