Kafka SSL服务配置及客户端使用(Linux+Pykafka)

内容:
  1: Kafka及Zookeeper快速安装配置及测试
  2: Kafka SSL服务端配置
  3: Kafka 客户端
    3.1: Linux下配置及测试
    3.2: Pykafka配置及测试
  4: 脚本
  5: 参考借鉴

1.Kafka及Zookeeper快速安装配置及测试
见此篇: https://www.cnblogs.com/wonglu/p/8687488.html

2.kafka SSL服务端配置

[root@kafka ~]# mkdir -p /root/round1
[root@kafka ~]# cd /root/round1
[root@kafka round1]# chmod +x test.sh
[root@kafka round1]# ./test.sh          #执行所有项
[root@kafka round1]# vim /opt/kafka_2.11-1.1.0/config/server.properties
listeners=PLAINTEXT://:9092,SSL://:9093
advertised.listeners=PLAINTEXT://本机IP:9092,SSL://本机IP:9093

test.sh 脚本见第4项;脚本生成的文件均在round1,必须在当前路径下执行脚本项,其他路径可能会出现文件未找到(file not found)。
3.kafka客户端
3.1Linux下配置及测试
拷贝round1文件夹下的client.keystore.jks 、client.truststore.jks、test.sh到其他机器上,三者必须在同一路径

配置:

[root@kafka round1]# scp client.keystore.jks client.truststore.jks test.sh root@client:/root/
[root@client ~]# ./test.sh   # 只需执行脚本第1、4项即可

测试:
从官网拉取kafka.tar.gz包,并解压

[root@client ~]# tar -zxvf kafka_2.11-1.1.0.tgz
[root@client ~]# /root/kafka_2.11-1.1.0/bin/kafka-console-producer.sh --broker-list <kafkaip>:9093 --topic test --producer.config /root/clientssl.properties              
>a
>b
>
[root@client ~]# /root/kafka_2.11-1.1.0/bin/kafka-console-consumer.sh --bootstrap-server <kafkaip>:9093 --topic test --from-beginning --consumer.config /root/clientssl.properties 
a
b

如上所示:生产消费均正常。

3.2Pykafka配置及测试
拷贝round1文件夹下的ca.crt、ca.key到D:\hhx\xxx\下
在这里插入图片描述

4.脚本:

#!/bin/bash
########################【Personality configuration item】#######################
BASE_DIR=`echo $PWD`"/"      
DOMAIN='machine03.zheng.com'
Password='leonzheng'
D_NAME='C=CN,ST=FJ,L=FZ,O=LEON,OU=LEON,CN=ZHENG.COM'
CA_DOMAIN='/C=CN/ST=FJ/L=FZ/O=LEON/OU=LEON/CN=ZHENG.COM'
####################################【 END】 ####################################

list_items(){
        echo "#####################################################" 
        echo "#                     kafka by ssl                  #"
        echo "#---------------------------------------------------#"
        echo "# 1.Env check         2.ssl tools      3.ssl server #"
        echo "# 4.ssl client        5.exit                        #"
        echo "#####################################################"
}
env_check(){
        java -version &> /dev/null
        if [ $? -ne 0 ];then
           echo -e "\033[31m java not found!\033[0m" 
           break
        fi
        keytool &> /dev/null
        if [ $? -ne 0 ];then
           echo -e "\033[31m java home not configure\033[0m" 
           break
        fi
        openssl version &> /dev/null
        if [ $? -ne 0 ];then
           echo -e "\033[31m openssl command not fount\033[0m" 
           break
        fi
        echo -e "\033[32m environment is OK\033[0m" 

}
create_ssl_tools(){
        echo $5
        echo "将生成本机进行双向验证的证书密钥工具"
# 生成服务器keystore(密钥和证书)
        keytool -keystore server.keystore.jks -alias $2 -validity 365 -keyalg RSA -storepass $3 -keypass $3 -genkey -dname "$4"
# 生成客户端keystore(密钥和证书)
        keytool -keystore client.keystore.jks -alias $2 -validity 365 -keyalg RSA -storepass $3 -keypass $3 -genkey -dname "$4"
# 创建CA证书
        openssl req -new -x509 -keyout ca.key -out ca.crt -days 365 -passout pass:$3 -subj "$5"
# 将CA证书导入到服务器truststore
        keytool -keystore server.truststore.jks -alias CARoot -import -file ca.crt -storepass $3
# 将CA证书导入到客户端truststore
        keytool -keystore client.truststore.jks -alias CARoot -import -file ca.crt -storepass $3
# 导出服务器证书
        keytool -keystore server.keystore.jks -alias $2 -certreq -file cert-file -storepass $3
        keytool -keystore client.keystore.jks -alias $2 -certreq -file client-cert-file -storepass $3
# 用CA证书给服务器证书签名
        openssl x509 -req -CA ca.crt -CAkey ca.key -in cert-file -out cert-signed -days 365 -CAcreateserial -passin pass:$3
        openssl x509 -req -CA ca.crt -CAkey ca.key -in client-cert-file -out client-cert-signed -days 365 -CAcreateserial -passin pass:$3
# 将CA证书导入服务器keystore
        keytool -keystore server.keystore.jks -alias CARoot -import -file ca.crt -storepass $3
        keytool -keystore client.keystore.jks -alias CARoot -import -file ca.crt -storepass $3
# 将已签名的服务器证书导入服务器keystore
        keytool -keystore server.keystore.jks -alias $2 -import -file cert-signed -storepass $3
        keytool -keystore client.keystore.jks -alias $2 -import -file client-cert-signed -storepass $3
}
config_ssl_client(){
        if [ ! -f $1client.truststore.jks ];then
                echo "$1client.truststore.jks" not found
                break
        fi
        if [ ! -f $1client.keystore.jks ];then
                echo "$1client.keystore.jks" not found
                break
        fi
        echo "security.protocol=SSL
ssl.truststore.location=$1client.truststore.jks
ssl.truststore.password=$3
ssl.keystore.location=$1client.keystore.jks
ssl.keystore.password=$3
ssl.key.password=$3" > $1"clientssl.properties"
}
config_ssl_server(){
        if [ ! -n "$6" ];then
                echo -e "\033[31m [Error]\033[0m \t <server.properties> file not found" 
                break
        fi
        if [ ! -f $1server.truststore.jks ];then
                echo "$1server.truststore.jks" not found
                break
        fi
        if [ ! -f $1server.keystore.jks ];then
                echo "$1server.keystore.jks" not found
                break
        fi
        echo "ssl.client.auth=required
ssl.keystore.location=$1server.keystore.jks
ssl.keystore.password=$3
ssl.key.password=$3
ssl.truststore.location=$1server.truststore.jks
ssl.truststore.password=$3
" >> $6
}
ssl_config(){
        echo "kafka服务器也是一个客户端,该步骤中,服务器证书将等同于客户端证书"
        create_ssl_tools $1 $2 $3 $4 $5
}
while :
do
  clear
  list_items
  read -p "Input your option:" option
  case $option in
    1) echo -e "\033[32m Starting env check......\033[0m"
       env_check
    ;;
    2) echo -e "\033[32m localhost ssl tools for server and client\033[0m"
       echo "相关证书密钥在该路径下产生:" $BASE_DIR
       ssl_config $BASE_DIR $DOMAIN $Password $D_NAME $CA_DOMAIN
    ;;
    3) echo -e "\033[32m [start]\033[0m \tconfig ssl server"
       config_file=`find / -name 'server.properties'`
       config_ssl_server $BASE_DIR $DOMAIN $Password $D_NAME $CA_DOMAIN $config_file
    ;;
    4) echo -e "\033[32m [start]\033[0m \tconfig ssl client"
       config_ssl_client $BASE_DIR $DOMAIN $Password $D_NAME $CA_DOMAIN
    ;;
    5) echo -e "\033[32m [Finish]\033[0m \tprogram exit."
       break
    ;;
    *) echo -e "\033[31m [Error]\033[0m \tOption not in items!"
  esac
  read -p "Input <enter> to continue..."
done

5.参考借鉴:
1.SSL加密和认证:https://www.orchome.com/171
2.SSL认证:https://blog.csdn.net/justry_deng/article/details/88383707
3.python生产者kafka报bytes错:https://blog.csdn.net/qq_35304570/article/details/81101395
4.Kafka Security 配置SSL: https://blog.csdn.net/difffate/article/details/53570344
5.集群配置及java端测试,附加ACL:https://blog.csdn.net/zbdba/article/details/52458654
6.云环境:http://www.voidcn.com/article/p-nwzohovi-bpy.html
7.双向认证:https://blog.csdn.net/hohoo1990/article/details/79110031

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章