45. 源代碼閱讀-RocketMQ-tools

一. 簡要介紹

RocketMQ-tools分爲3部分

admin
command
monitor

下面一一介紹

二. admin

提供了管理操作接口

三. command

command提供了命令行控制MQ的一些方法。

啓動方法

進入RocketMQ安裝目錄，執行sh bin/mqadmin

有幾個參數的執行方法

1. sh bin/mqadmin

打印命令提示，如下：

The most commonly used mqadmin commands are:
   updateTopic          Update or create topic
   deleteTopic          Delete topic from broker and NameServer.
   updateSubGroup       Update or create subscription group
   deleteSubGroup       Delete subscription group from broker.
   updateBrokerConfig   Update broker's config
   updateTopicPerm      Update topic perm
   topicRoute           Examine topic route info
   topicStatus          Examine topic Status info
   topicClusterList     get cluster info for topic
   brokerStatus         Fetch broker runtime status data
   queryMsgById         Query Message by Id
   queryMsgByKey        Query Message by Key
   queryMsgByUniqueKey  Query Message by Unique key
   queryMsgByOffset     Query Message by offset
   queryMsgByUniqueKey  Query Message by Unique key
   printMsg             Print Message Detail
   printMsgByQueue      Print Message Detail
   sendMsgStatus        send msg to broker.
     ...

2. sh bin/mqadmin help xxx

打印某個命令的提示參數，比如sh bin/mqadmin helop updatetopic(不需要區分大小寫)

[root@bogon apache-rocketmq]# sh bin/mqadmin help updatetopic
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
usage: mqadmin updateTopic [-b <arg>] [-c <arg>] [-h] [-n <arg>] [-o <arg>] [-p <arg>] [-r <arg>] [-s <arg>]
       -t <arg> [-u <arg>] [-w <arg>]
 -b,--brokerAddr <arg>       create topic to which broker
 -c,--clusterName <arg>      create topic to which cluster
 -h,--help                   Print help
 -n,--namesrvAddr <arg>      Name server address list, eg: 192.168.0.1:9876;192.168.0.2:9876
 -o,--order <arg>            set topic's order(true|false
 -p,--perm <arg>             set topic's permission(2|4|6), intro[2:W 4:R; 6:RW]
 -r,--readQueueNums <arg>    set read queue nums
 -s,--hasUnitSub <arg>       has unit sub (true|false
 -t,--topic <arg>            topic name
 -u,--unit <arg>             is unit topic (true|false
 -w,--writeQueueNums <arg>   set write queue nums

3. sh bin/mqadmin xx 具體執行某個命令

比如 sh bin/mqadmin updatetopic -x xxx

源代碼分析

首先來看bin/mqadmin這個啓動腳本

1. bin/mqadmin

if [ -z "$ROCKETMQ_HOME" ] ; then
  ## resolve links - $0 may be a link to maven's home
  PRG="$0"

  # need this for relative symlinks
  while [ -h "$PRG" ] ; do
    ls=`ls -ld "$PRG"`
    link=`expr "$ls" : '.*-> \(.*\)$'`
    if expr "$link" : '/.*' > /dev/null; then
      PRG="$link"
    else
      PRG="`dirname "$PRG"`/$link"
    fi
  done

  saveddir=`pwd`

  ROCKETMQ_HOME=`dirname "$PRG"`/..

  # make it fully qualified
  ROCKETMQ_HOME=`cd "$ROCKETMQ_HOME" && pwd`

  cd "$saveddir"
fi

export ROCKETMQ_HOME

sh ${ROCKETMQ_HOME}/bin/tools.sh org.apache.rocketmq.tools.command.MQAdminStartup $@

前面在設置ROCKETMQ_HOME環境變量，然後啓動MQAdminStartup這個類。

2. MQAdminStartup

public class MQAdminStartup {
    protected static List<SubCommand> subCommandList = new ArrayList<SubCommand>();

    public static void main(String[] args) {
        main0(args, null);
    }

    public static void main0(String[] args, RPCHook rpcHook) {
        ...

        initCommand();

        try {
            initLogback();
            switch (args.length) {
                case 0:
                    printHelp();
                    break;
                case 2:
                    if (args[0].equals("help")) {
                        SubCommand cmd = findSubCommand(args[1]);
                        if (cmd != null) {
                            Options options = ServerUtil.buildCommandlineOptions(new Options());
                            options = cmd.buildCommandlineOptions(options);
                            if (options != null) {
                                ServerUtil.printCommandLineHelp("mqadmin " + cmd.commandName(), options);
                            }
                        } else {
                            System.out.printf("The sub command \'" + args[1] + "\' not exist.%n");
                        }
                        break;
                    }
                case 1:
                default:
                    SubCommand cmd = findSubCommand(args[0]);
                    if (cmd != null) {
                        ....

                        cmd.execute(commandLine, options, rpcHook);
                    } else {
                        System.out.printf("The sub command \'" + args[0] + "\' not exist.%n");
                    }
                    break;
            }
        } 
                ...
    }

    public static void initCommand() {
        initCommand(new UpdateTopicSubCommand());
        initCommand(new DeleteTopicSubCommand());
        initCommand(new UpdateSubGroupSubCommand());
        initCommand(new DeleteSubscriptionGroupCommand());
        initCommand(new UpdateBrokerConfigSubCommand());
        initCommand(new UpdateTopicPermSubCommand());

        initCommand(new TopicRouteSubCommand());
        initCommand(new TopicStatusSubCommand());
        initCommand(new TopicClusterSubCommand());

        initCommand(new BrokerStatusSubCommand());
        initCommand(new QueryMsgByIdSubCommand());
        initCommand(new QueryMsgByKeySubCommand());
        initCommand(new QueryMsgByUniqueKeySubCommand());
        initCommand(new QueryMsgByOffsetSubCommand());
        initCommand(new QueryMsgByUniqueKeySubCommand());
        initCommand(new PrintMessageSubCommand());
        initCommand(new PrintMessageByQueueCommand());
        initCommand(new SendMsgStatusCommand());
        initCommand(new BrokerConsumeStatsSubCommad());

        initCommand(new ProducerConnectionSubCommand());
        initCommand(new ConsumerConnectionSubCommand());
        initCommand(new ConsumerProgressSubCommand());
        initCommand(new ConsumerStatusSubCommand());
        initCommand(new CloneGroupOffsetCommand());

        initCommand(new ClusterListSubCommand());
        initCommand(new TopicListSubCommand());

        initCommand(new UpdateKvConfigCommand());
        initCommand(new DeleteKvConfigCommand());

        initCommand(new WipeWritePermSubCommand());
        initCommand(new ResetOffsetByTimeCommand());

        initCommand(new UpdateOrderConfCommand());
        initCommand(new CleanExpiredCQSubCommand());
        initCommand(new CleanUnusedTopicCommand());

        initCommand(new StartMonitoringSubCommand());
        initCommand(new StatsAllSubCommand());

        initCommand(new AllocateMQSubCommand());

        initCommand(new CheckMsgSendRTCommand());
        initCommand(new CLusterSendMsgRTCommand());

        initCommand(new GetNamesrvConfigCommand());
        initCommand(new UpdateNamesrvConfigCommand());
        initCommand(new GetBrokerConfigCommand());

        initCommand(new QueryConsumeQueueCommand());
    }
        ...

邏輯很簡單，

首先初始化可以支持的Command - initCommand
initLogback根據logback_tools.xml初始化，主要是日誌系統。
然後根據傳入過來的參數，選擇不同的執行分支，也就是上面舉例的參數。

3. 某個指令執行過程

以updatetopic爲例

....
cmd.execute(commandLine, options, rpcHook);

進入UpdateTopicSubCommand.execute(xx)方法

@Override
    public void execute(final CommandLine commandLine, final Options options,
        RPCHook rpcHook) throws SubCommandException {
        ...
        try {
            TopicConfig topicConfig = new TopicConfig();
            topicConfig.setReadQueueNums(8);
            topicConfig.setWriteQueueNums(8);
            topicConfig.setTopicName(commandLine.getOptionValue('t').trim());

            // readQueueNums
            if (commandLine.hasOption('r')) {
                topicConfig.setReadQueueNums(Integer.parseInt(commandLine.getOptionValue('r').trim()));
            }

            // writeQueueNums
            if (commandLine.hasOption('w')) {
                topicConfig.setWriteQueueNums(Integer.parseInt(commandLine.getOptionValue('w').trim()));
            }

            ...

            if (commandLine.hasOption('b')) {
                String addr = commandLine.getOptionValue('b').trim();

                defaultMQAdminExt.start();
                defaultMQAdminExt.createAndUpdateTopicConfig(addr, topicConfig);

                ...
                return;

            } else if (commandLine.hasOption('c')) {
                String clusterName = commandLine.getOptionValue('c').trim();

                defaultMQAdminExt.start();

                Set<String> masterSet =
                    CommandUtil.fetchMasterAddrByClusterName(defaultMQAdminExt, clusterName);
                for (String addr : masterSet) {
                    defaultMQAdminExt.createAndUpdateTopicConfig(addr, topicConfig);
                    System.out.printf("create topic to %s success.%n", addr);
                }

                ...
                }

                System.out.printf("%s", topicConfig);
                return;
            }

            ServerUtil.printCommandLineHelp("mqadmin " + this.commandName(), options);
        } catch (Exception e) {
            throw new SubCommandException(this.getClass().getSimpleName() + " command failed", e);
        } finally {
            defaultMQAdminExt.shutdown();
        }
    }

execute的邏輯就是獲取傳入的參數，-t是必須的，然後還必須帶上-b或者-c纔會執行。
如果沒有執行，那麼會打印updatetopic的參數列表。
-b代表某個broker, -c代表集羣

所以大概格式是： sh ./bin/mqadmin updatetopic -t topictest -b xxx

其他參數是可選的。

4. 調用執行

如果確定要更新topic,那麼就會調用RocketMQ-cli裏面的接口進行更新。
更新成功會打印更新成功輸出，否則會報異常。

附上所有命令的操作參數
https://www.cnblogs.com/zyguo/p/4962425.html

四. monitor

監控相關

啓動類是參數是startMonitoring

sh bin/mqadmin startMonitoring

代碼如下：

public class StartMonitoringSubCommand implements SubCommand {
    private final Logger log = ClientLogger.getLog();

    @Override
    public String commandName() {
        return "startMonitoring";
    }

    @Override
    public String commandDesc() {
        return "Start Monitoring";
    }

    @Override
    public Options buildCommandlineOptions(Options options) {
        return options;
    }

    @Override
    public void execute(CommandLine commandLine, Options options, RPCHook rpcHook) throws SubCommandException {
        try {
            MonitorService monitorService =
                new MonitorService(new MonitorConfig(), new DefaultMonitorListener(), rpcHook);

            monitorService.start();
        } catch (Exception e) {
            throw new SubCommandException(this.getClass().getSimpleName() + " command failed", e);
        }
    }
}

五. 命令執行和相關代碼分析

獲取所有的topic -- topiclist

1. 命令如下

sh bin/mqadmin topiclist -n localhost:9876

2. 結果如下：

huangrongweideMacBook-Pro:apache-rocketmq huangrongwei$ sh bin/mqadmin topiclist -n localhost:9876
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
%RETRY%lalaase_rename_unique_group_name_4
%RETRY%please_rename_unique_group_name_4
huangrongweideMacBook-Pro.local
%RETRY%bybybse_rename_unique_group_name_4
BenchmarkTest
OFFSET_MOVED_EVENT
%RETRY%phihomebse_rename_unique_group_name_4
wangyuan.freecomm-networks.com
TBW102
SELF_TEST_TOPIC
mmmzzz
DefaultCluster

3. 參數

-n表示要顯示連接到的namesrv
-c

4. 實現這個功能的類

rocketmq/tools/src/main/java/org/apache/rocketmq/tools/command/topic/TopicListSubCommand.java

5. 代碼分析

a. 入口類是MQAdminStartup.main方法，走的是case 1分支，如下：

public static void main0(String[] args, RPCHook rpcHook) {
        ...

        initCommand();

        try {
                                ...
                case 1:
                default:
                    SubCommand cmd = findSubCommand(args[0]);
                    if (cmd != null) {
                        ...

                        if (commandLine.hasOption('n')) {
                            String namesrvAddr = commandLine.getOptionValue('n');
                            System.setProperty(MixAll.NAMESRV_ADDR_PROPERTY, namesrvAddr);
                        }

                        cmd.execute(commandLine, options, rpcHook);
                    } else {
                        System.out.printf("The sub command \'" + args[0] + "\' not exist.%n");
                    }
                    break;
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
}

如果commandLine包含n參數，也就是如果輸入命令有帶-n的參數，比如sh bin/mqadmin topiclist -n localhost:9876。那麼把它帶的參數值拿出來，然後設置到SystemProperty裏面。

System.setProperty(MixAll.NAMESRV_ADDR_PROPERTY, namesrvAddr);

這樣做是爲了其他地方可以拿這個參數，也就是RocketMQ傳遞namesrv address的時候，不是用普通的參數傳遞，而是用系統屬性。

這樣做有好處也有不好的地方吧，好處就是省心，一個地址設置了，很多地方都有可以得到這個值。不需要每個參數傳來傳去。不好的地方就是如果設置的地方多了，就容易混淆，互相影響。

然後執行command的execute方法，也就是TopicListSubCommand.execute方法。

cmd.execute(commandLine, options, rpcHook);

所有的command都實現了SubCommand接口。

public class TopicListSubCommand implements SubCommand {}

SubCommand提供了幾個接口方法

public interface SubCommand {
    String commandName(); //命令名稱，用戶用這個名稱調用這個命令，不區分大小寫，比如topiclist。

    String commandDesc(); //命令描述

    Options buildCommandlineOptions(final Options options); //參數構成

    void execute(final CommandLine commandLine, final Options options, RPCHook rpcHook) throws SubCommandException;//命令執行
}

b. TopicListSubCommand.execute

public void execute(final CommandLine commandLine, final Options options,
        RPCHook rpcHook) throws SubCommandException {
        ...

        try {
            ...
            defaultMQAdminExt.start();
            if (commandLine.hasOption('c')) {
                ClusterInfo clusterInfo = defaultMQAdminExt.examineBrokerClusterInfo();

                ...
                TopicList topicList = defaultMQAdminExt.fetchAllTopicList();
                ...
            } else {
                ...
                TopicList topicList = defaultMQAdminExt.fetchAllTopicList();
                for (String topic : topicList.getTopicList()) {
                    System.out.printf("%s%n", topic);
                }
            }
        } catch (Exception e) {
            throw new SubCommandException(this.getClass().getSimpleName() + " command failed", e);
        } finally {
            defaultMQAdminExt.shutdown();
        }
    }

這個類主要就是根據傳入的參數是否有-c 或者-n 獲取相應的topic，然後打印出來。

以-n爲例，它的邏輯就是去namesrv獲取所有的topic，然後打印出來。