Docker部署RocketMQ集羣

一、概述

Apache RocketMQ是阿里開源的一款高性能、高吞吐量、隊列模型的消息中間件的分佈式消息中間件。

關於RocketMQ集羣架構的詳細介紹，請參考鏈接：https://blog.csdn.net/Weixiaohuai/article/details/123607472

 

二、下載鏡像

docker pull apache/rocketmq:4.9.4
docker pull apacherocketmq/rocketmq-dashboard

主要用到了2個鏡像，第1個用來部署NameServer和Broker，第2個用來管理rocketmq

 

三、部署

本文主要採用一臺服務器來部署RocketMQ集羣，集羣部署模式採用多master模式，也就是2個NameServer和2個Broker

刷盤機制採用異步複製，異步刷盤

 

服務器配置：

操作系統：centos 7.6

硬件配置：2核4g

ip地址：192.168.137.138

 

創建目錄

mkdir -p /opt/rocketmq/logs/nameserver-a
mkdir -p /opt/rocketmq/logs/nameserver-b
mkdir -p /opt/rocketmq/logs/broker-a
mkdir -p /opt/rocketmq/logs/broker-b
mkdir -p /opt/rocketmq/store/broker-a
mkdir -p /opt/rocketmq/store/broker-b
mkdir -p /opt/rocketmq/broker-a/
mkdir -p /opt/rocketmq/broker-b/

設置權限

chmod 777 -R /opt/rocketmq/logs/*
chmod 777 -R /opt/rocketmq/store/*

注意：這裏如果不設置權限，會導致docker寫入文件失敗，導致rocketmq啓動異常。

 

創建broker.conf

修改第一個節點配置

vim /opt/rocketmq/broker-a/broker-a.conf

內容如下：

brokerClusterName = rocketmq-cluster
brokerName = broker-a
brokerId = 0
#這個很有講究 如果是正式環境 這裏一定要填寫內網地址（安全）
#如果是用於測試或者本地這裏建議要填外網地址，因爲你的本地代碼是無法連接到阿里雲內網，只能連接外網。
brokerIP1 = 192.168.137.138
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
# 內網的(阿里雲有內網IP和外網IP)
namesrvAddr=192.168.137.138:9876;192.168.137.138:9877
autoCreateTopicEnable=true
#Broker 對外服務的監聽端口,
listenPort = 10911
#Broker角色
#- ASYNC_MASTER 異步複製Master
#- SYNC_MASTER 同步雙寫Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盤方式
#- ASYNC_FLUSH 異步刷盤
#- SYNC_FLUSH 同步刷盤
flushDiskType=ASYNC_FLUSH

注意：brokerIP1，namesrvAddr，這2個參數，一般配置爲內網ip，提供內網訪問。如果需要公網訪問，這裏一定要配置公網ip，否則無法訪問。

 

修改第二個節點配置

vim /opt/rocketmq/broker-b/broker-b.conf

內容如下：

brokerClusterName = rocketmq-cluster
brokerName = broker-b
brokerId = 0
#這個很有講究 如果是正式環境 這裏一定要填寫內網地址（安全）
#如果是用於測試或者本地這裏建議要填外網地址，因爲你的本地代碼是無法連接到阿里雲內網，只能連接外網。
brokerIP1 = 192.168.137.138
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
# 內網的(阿里雲有內網IP和外網IP)
namesrvAddr=192.168.137.138:9876;192.168.137.138:9877
autoCreateTopicEnable=true
#Broker 對外服務的監聽端口,
listenPort = 10912
#Broker角色
#- ASYNC_MASTER 異步複製Master
#- SYNC_MASTER 同步雙寫Master
#- SLAVE
brokerRole=ASYNC_MASTER
#刷盤方式
#- ASYNC_FLUSH 異步刷盤
#- SYNC_FLUSH 同步刷盤
flushDiskType=ASYNC_FLUSH

注意：brokerName，listenPort這2個參數和第一個節點不一樣，其他內容和第一個節點配置一樣

 

編寫 docker-compose.yml

vim /opt/rocketmq/docker-compose.yml

內容如下：

version: '3.5'
services:
  rmqnamesrv-a:
    image: apache/rocketmq:4.9.4
    container_name: rmqnamesrv-a
    ports:
      - 9876:9876
    volumes:
      #- /opt/rocketmq/logs/nameserver-a:/home/rocketmq/logs
      - /opt/rocketmq/broker-b/broker-b.conf:/home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    command: sh mqnamesrv
    networks:
      rmq:
        aliases:
          - rmqnamesrv-a

  rmqnamesrv-b:
    image: apache/rocketmq:4.9.4
    container_name: rmqnamesrv-b
    ports:
      - 9877:9876
    volumes:
      #- /opt/rocketmq/logs/nameserver-b:/home/rocketmq/logs
      - /opt/rocketmq/broker-b/broker-b.conf:/home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    command: sh mqnamesrv
    networks:
      rmq:
        aliases:
          - rmqnamesrv-b

  rmqbroker-a:
    image: apache/rocketmq:4.9.4
    container_name: rmqbroker-a
    ports:
      - 10911:10911
    volumes:
      #- /opt/rocketmq/logs/broker-a/logs:/home/rocketmq/logs
      #- /opt/rocketmq/store/broker-a/store:/home/rocketmq/store
      - /opt/rocketmq/broker-a/broker-a.conf:/home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    environment:
      TZ: Asia/Shanghai
      NAMESRV_ADDR: "rmqnamesrv-a:9876"
      JAVA_OPTS: " -Duser.home=/opt"
      JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m"
    command: sh mqbroker -c /home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    links:
      - rmqnamesrv-a:rmqnamesrv-a
      - rmqnamesrv-b:rmqnamesrv-b
    networks:
      rmq:
        aliases:
          - rmqbroker-a

  rmqbroker-b:
    image: apache/rocketmq:4.9.4
    container_name: rmqbroker-b
    ports:
      - 10912:10912
    volumes:
      #- /opt/rocketmq/logs/broker-b/logs:/home/rocketmq/logs
      #- /opt/rocketmq/store/broker-b/store:/home/rocketmq/store
      - /opt/rocketmq/broker-b/broker-b.conf:/home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    environment:
      TZ: Asia/Shanghai
      NAMESRV_ADDR: "rmqnamesrv-b:9877"
      JAVA_OPTS: " -Duser.home=/opt"
      JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m"
    command: sh mqbroker -c /home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    links:
      - rmqnamesrv-a:rmqnamesrv-a
      - rmqnamesrv-b:rmqnamesrv-b
    networks:
      rmq:
        aliases:
          - rmqbroker-b
  rmqconsole:
    image: apacherocketmq/rocketmq-dashboard
    container_name: rmqconsole
    ports:
      - 8087:8080
    environment:
      JAVA_OPTS: -Drocketmq.namesrv.addr=rmqnamesrv-a:9876;rmqnamesrv-b:9877 -Dcom.rocketmq.sendMessageWithVIPChannel=false -Drocketmq.config.accessKey=rocketmq2 -Drocketmq.config.secretKey=12345678
    volumes:
      - /opt/rocketmq/console-ng/data:/tmp/rocketmq-console/data
    networks:
      rmq:
        aliases:
          - rmqconsole
networks:
  rmq:
    name: rmq
    driver: bridge

 

最後一步，執行docker-compose

cd /opt/rocketmq
docker-compose up -d

 

查看進程，使用命令docker-compose ps

# docker-compose ps
    Name                  Command               State                Ports              
----------------------------------------------------------------------------------------
rmqbroker-a    sh mqbroker -c /home/rocke ...   Up      10909/tcp,                      
                                                        0.0.0.0:10911->10911/tcp,       
                                                        10912/tcp, 9876/tcp             
rmqbroker-b    sh mqbroker -c /home/rocke ...   Up      10909/tcp, 10911/tcp,           
                                                        0.0.0.0:10912->10912/tcp,       
                                                        9876/tcp                        
rmqconsole     sh -c java $JAVA_OPTS -jar ...   Up      0.0.0.0:8087->8080/tcp          
rmqnamesrv-a   sh mqnamesrv                     Up      10909/tcp, 10911/tcp, 10912/tcp,
                                                        0.0.0.0:9876->9876/tcp          
rmqnamesrv-b   sh mqnamesrv                     Up      10909/tcp, 10911/tcp, 10912/tcp,
                                                        0.0.0.0:9877->9876/tcp 

確保State狀態爲up

 

訪問管理平臺

http://192.168.137.138:8087

點擊右上角切換語言爲簡體中文，點擊集羣，效果如下：

 

看到這個圖，說明集羣啓動正常。

 

設置持久化配置

上面的 docker-compose.yml，註釋掉了一些文件掛載操作，因爲新版本的rocketmq無法啓動創建文件，直接掛載一個空目錄，會啓動報錯。

 

複製docker裏面的文件

docker cp rmqnamesrv-a:/home/rocketmq/logs /opt/rocketmq/logs/nameserver-a
docker cp rmqnamesrv-b:/home/rocketmq/logs /opt/rocketmq/logs/nameserver-b
docker cp rmqbroker-a:/home/rocketmq/logs /opt/rocketmq/logs/broker-a
docker cp rmqbroker-a:/home/rocketmq/store /opt/rocketmq/store/broker-a
docker cp rmqbroker-b:/home/rocketmq/logs /opt/rocketmq/logs/broker-b
docker cp rmqbroker-b:/home/rocketmq/store /opt/rocketmq/store/broker-b

設置權限，否則會寫入文件失敗，造成啓動失敗。

chmod 777 -R /opt/rocketmq/logs/*
chmod 777 -R /opt/rocketmq/store/*

 

關閉相關docker進程

docker-compose down

 

修改docker-compose.yml，開啓持久化配置

version: '3.5'
services:
  rmqnamesrv-a:
    image: apache/rocketmq:4.9.4
    container_name: rmqnamesrv-a
    ports:
      - 9876:9876
    volumes:
      - /opt/rocketmq/logs/nameserver-a:/home/rocketmq/logs
      - /opt/rocketmq/broker-b/broker-b.conf:/home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    command: sh mqnamesrv
    networks:
      rmq:
        aliases:
          - rmqnamesrv-a

  rmqnamesrv-b:
    image: apache/rocketmq:4.9.4
    container_name: rmqnamesrv-b
    ports:
      - 9877:9876
    volumes:
      - /opt/rocketmq/logs/nameserver-b:/home/rocketmq/logs
      - /opt/rocketmq/broker-b/broker-b.conf:/home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    command: sh mqnamesrv
    networks:
      rmq:
        aliases:
          - rmqnamesrv-b

  rmqbroker-a:
    image: apache/rocketmq:4.9.4
    container_name: rmqbroker-a
    ports:
      - 10911:10911
    volumes:
      - /opt/rocketmq/logs/broker-a/logs:/home/rocketmq/logs
      - /opt/rocketmq/store/broker-a/store:/home/rocketmq/store
      - /opt/rocketmq/broker-a/broker-a.conf:/home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    environment:
      TZ: Asia/Shanghai
      NAMESRV_ADDR: "rmqnamesrv-a:9876"
      JAVA_OPTS: " -Duser.home=/opt"
      JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m"
    command: sh mqbroker -c /home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    links:
      - rmqnamesrv-a:rmqnamesrv-a
      - rmqnamesrv-b:rmqnamesrv-b
    networks:
      rmq:
        aliases:
          - rmqbroker-a

  rmqbroker-b:
    image: apache/rocketmq:4.9.4
    container_name: rmqbroker-b
    ports:
      - 10912:10912
    volumes:
      - /opt/rocketmq/logs/broker-b/logs:/home/rocketmq/logs
      - /opt/rocketmq/store/broker-b/store:/home/rocketmq/store
      - /opt/rocketmq/broker-b/broker-b.conf:/home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    environment:
      TZ: Asia/Shanghai
      NAMESRV_ADDR: "rmqnamesrv-b:9877"
      JAVA_OPTS: " -Duser.home=/opt"
      JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m"
    command: sh mqbroker -c /home/rocketmq/rocketmq-4.9.4/conf/broker.conf
    links:
      - rmqnamesrv-a:rmqnamesrv-a
      - rmqnamesrv-b:rmqnamesrv-b
    networks:
      rmq:
        aliases:
          - rmqbroker-b
  rmqconsole:
    image: apacherocketmq/rocketmq-dashboard
    container_name: rmqconsole
    ports:
      - 8087:8080
    environment:
      JAVA_OPTS: -Drocketmq.namesrv.addr=rmqnamesrv-a:9876;rmqnamesrv-b:9877 -Dcom.rocketmq.sendMessageWithVIPChannel=false -Drocketmq.config.accessKey=rocketmq2 -Drocketmq.config.secretKey=12345678
    volumes:
      - /opt/rocketmq/console-ng/data:/tmp/rocketmq-console/data
    networks:
      rmq:
        aliases:
          - rmqconsole
networks:
  rmq:
    name: rmq
    driver: bridge

 

啓動相關docker進程

docker-compose up -d

 

等待30秒，再次訪問管理後臺，如果數據能顯示，表示正常。

 

測試

點擊主題，默認是空的

點擊新增

 

 選擇集羣，BROKER_NAME，輸入主題名test

 

點擊提交後，會出現一條數據，點擊發送消息

 

 點擊提交

 

 提示成功

 

 點擊消息，選擇主題test，點擊搜索，就可以看到發送記錄了。

 

 

 

本文參考鏈接：

https://www.136.la/jingpin/show-176132.html