1.各個組件啓動源碼、框架結構

1.1 NameServer啓動

NamesrvStartup#main

1）NamesrvController controller = createNamesrvController(args);創建controller
1-1）檢測命令行參數
1-2）創建兩個核心配置：NamesrvConfig和NettyServerConfig
1-3）解析-c 和 -p 參數，賦值到上面兩個配置中
1-4）ROCKETMQ_HOME環境變量檢測
1-5）controller = new NamesrvController(namesrvConfig, nettyServerConfig);創建controller
1-6） controller.getConfiguration().registerConfig(properties);註冊一下所有的配置
2）start(controller);啓動controller
2-1）controller.initialize();初始化
A）this.kvConfigManager.load();加載KV配置
B）this.remotingServer = new NettyRemotingServer(this.nettyServerConfig, this.brokerHousekeepingService);創建NettyServer網絡處理對象
C） this.remotingExecutor = Executors.newFixedThreadPool()Netty服務器的工作線程池
D）this.registerProcessor();註冊NameServer的Processor 註冊到RemotingServer中。
E）NamesrvController.this.routeInfoManager.scanNotActiveBroker();定時任務：每間隔10S掃描一次Broker，移除不活躍的Broker
F）NamesrvController.this.kvConfigManager.printAllPeriodically();定時任務：每間隔10min打印一次KV配置
2-2）Runtime.getRuntime().addShutdownHook()服務關閉鉤子，在服務正常關閉時執行。
2-3） controller.start();啓動服務
A）this.remotingServer.start();啓動remotingServer。
B）不爲null也啓動this.fileWatchService.start();

NameServer的核心作用：

一是維護Broker的服務地址並進行及時的更新。
二是給Producer和Consumer提供服務獲取Broker列表。

1.2 Broker啓動

BrokerStartup#main

1）createBrokerController(args)創建controller
1-1）檢測命令行參數
1-2）創建四個核心配置：BrokerConfig、NettyServerConfig、NettyClientConfig和MessageStoreConfig
1-3）解析-c 參數，賦值到上面四個配置中
1-4）ROCKETMQ_HOME環境變量檢測
1-5）處理NamesrcAddr
1-6）通過brokerId判斷主從：ASYNC_MASTER、SYNC_MASTER、SLAVE；Dledger集羣的所有Broker節點ID都是-1
1-7）解析-p和-m參數，賦值到上面四個配置中
1-8） controller = new BrokerController()，創建核心的BrokerController
1-9）controller.getConfiguration().registerConfig(properties);註冊配置
1-10）controller.initialize();初始化BrokerController。
A）加載磁盤上的配置信息（json）。topicConfigManager、consumerOffsetManager、subscriptionGroupManager、consumerFilterManager
B）構建消息存儲管理組件DefaultMessageStore，外層還會包裝插件AbstractPluginMessageStore
C）加載磁盤文件this.messageStore.load();
D）this.remotingServer = new NettyRemotingServer()
E）this.fastRemotingServer = new NettyRemotingServer()這個fastRemotingServer與RemotingServer功能基本差不多，處理VIP端口請求
F） this.sendMessageExecutor發送消息的線程池
G） this.pullMessageExecutor處理consumer的pull請求的線程池
H） this.replyMessageExecutor回覆消息的線程池
I）this.queryMessageExecutor查詢消息的線程池
J）this.adminBrokerExecutor
K）this.clientManageExecutor管理客戶端的線程池
L） this.heartbeatExecutor心跳請求線程池
M）this.endTransactionExecutor
N） this.consumerManageExecutor
O）this.registerProcessor();Broker註冊Processor
P）BrokerController.this.getBrokerStats().record();定時進行broker統計的任務
Q）BrokerController.this.consumerOffsetManager.persist();定時進行consumer消費Offset持久化到磁盤的任務
R）BrokerController.this.consumerFilterManager.persist();對consumer的filter過濾器進行持久化的任務。這裏可以看到，消費者的filter是被下推到了Broker來執行的。
S）BrokerController.this.protectBroker();定時進行broker保護
T）BrokerController.this.printWaterMark();定時打印水位線 U）BrokerController.this.getMessageStore().dispatchBehindByte
s()定時進行落後commitlog分發的任務
V）this.brokerOuterAPI.updateNameServerAddressList(this.brokerConfig.getNamesrvAddr());或者BrokerController.this.brokerOuterAPI.fetchNameServerAddr();設置NameServer的地址列表。可以從配置加載，也可以發遠程請求加載
W）initialTransaction();
X）initialAcl(); 權限控制
Y）initialRpcHooks();
1-11）Runtime.getRuntime().addShutdownHook()服務關閉鉤子，在服務正常關閉時執行。
2）start()啓動controller
A）this.messageStore.start();存儲組件，這裏啓動服務主要是爲了將CommitLog的寫入事件分發給ComsumeQueue和IndexFile
B）this.remotingServer.start();以及this.fastRemotingServer.start();
C）this.fileWatchService.start();
D） this.brokerOuterAPI.start();Broker的brokerOuterAPI可以理解爲一個Netty客戶端，往外發請求的組件。例如發送心跳
E）this.pullRequestHoldService.start();長輪詢請求暫存服務
F）this.clientHousekeepingService.start();
G）this.filterServerManager.start();
H）BrokerController.this.registerBrokerAll(true, false, brokerConfig.isForceRegister());Broker核心的心跳註冊任務
I）this.brokerStatsManager.start();
J） this.brokerFastFailure.start();

1.3 Netty網絡框架

Broker端的NettyRemotingServer：

NettyRemotingServer#start

    public void start() {
        this.defaultEventExecutorGroup = new DefaultEventExecutorGroup(
            nettyServerConfig.getServerWorkerThreads(),
            new ThreadFactory() {

                private AtomicInteger threadIndex = new AtomicInteger(0);

                @Override
                public Thread newThread(Runnable r) {
                    return new Thread(r, "NettyServerCodecThread_" + this.threadIndex.incrementAndGet());
                }
            });

        prepareSharableHandlers();
        //K1 Netty服務啓動的核心流程
        ServerBootstrap childHandler =
            this.serverBootstrap.group(this.eventLoopGroupBoss, this.eventLoopGroupSelector)
                .channel(useEpoll() ? EpollServerSocketChannel.class : NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .option(ChannelOption.SO_REUSEADDR, true)
                .option(ChannelOption.SO_KEEPALIVE, false)
                .childOption(ChannelOption.TCP_NODELAY, true)
                .childOption(ChannelOption.SO_SNDBUF, nettyServerConfig.getServerSocketSndBufSize())
                .childOption(ChannelOption.SO_RCVBUF, nettyServerConfig.getServerSocketRcvBufSize())
                .localAddress(new InetSocketAddress(this.nettyServerConfig.getListenPort()))
                .childHandler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    public void initChannel(SocketChannel ch) throws Exception {
                        //Netty的核心服務流程，encoder和decoder，二進制傳輸協議。
                        //RocketMQ中的二進制傳輸協議比較複雜，是否能按照JSON自定義二進制協議？
                        //serverHandler負責最關鍵的網絡請求處理。
                        ch.pipeline()
                            .addLast(defaultEventExecutorGroup, HANDSHAKE_HANDLER_NAME, handshakeHandler)
                            .addLast(defaultEventExecutorGroup,
                                encoder,
                                new NettyDecoder(),
                                new IdleStateHandler(0, 0, nettyServerConfig.getServerChannelMaxIdleTimeSeconds()),
                                connectionManageHandler,
                                serverHandler
                            );
                    }
                });

        if (nettyServerConfig.isServerPooledByteBufAllocatorEnable()) {
            childHandler.childOption(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT);
        }
        //開始Socket監聽
        try {
            ChannelFuture sync = this.serverBootstrap.bind().sync();
            InetSocketAddress addr = (InetSocketAddress) sync.channel().localAddress();
            this.port = addr.getPort();
        } catch (InterruptedException e1) {
            throw new RuntimeException("this.serverBootstrap.bind().sync() InterruptedException", e1);
        }

        if (this.channelEventListener != null) {
            this.nettyEventExecutor.start();
        }
        //每秒清理過期的異步請求暫存結果。
        this.timer.scheduleAtFixedRate(new TimerTask() {

            @Override
            public void run() {
                try {
                    NettyRemotingServer.this.scanResponseTable();
                } catch (Throwable e) {
                    log.error("scanResponseTable exception", e);
                }
            }
        }, 1000 * 3, 1000);
    }

核心是NettyServerHandler

    @ChannelHandler.Sharable
    class NettyServerHandler extends SimpleChannelInboundHandler<RemotingCommand> {

        @Override
        protected void channelRead0(ChannelHandlerContext ctx, RemotingCommand msg) throws Exception {
            processMessageReceived(ctx, msg);
        }
    }

    public void processMessageReceived(ChannelHandlerContext ctx, RemotingCommand msg) throws Exception {
        final RemotingCommand cmd = msg;
        if (cmd != null) {
            switch (cmd.getType()) {
                case REQUEST_COMMAND:
                    processRequestCommand(ctx, cmd);
                    break;
                case RESPONSE_COMMAND:
                    processResponseCommand(ctx, cmd);
                    break;
                default:
                    break;
            }
        }
    }

    public void processRequestCommand(final ChannelHandlerContext ctx, final RemotingCommand cmd) {
        final Pair<NettyRequestProcessor, ExecutorService> matched = this.processorTable.get(cmd.getCode());

核心的業務處理是在processorTable，其初始化是在BrokerController#registerProcessor

    public void registerProcessor() {
        /**
         * SendMessageProcessor
         */
        SendMessageProcessor sendProcessor = new SendMessageProcessor(this);
        sendProcessor.registerSendMessageHook(sendMessageHookList);
        sendProcessor.registerConsumeMessageHook(consumeMessageHookList);

        this.remotingServer.registerProcessor(RequestCode.SEND_MESSAGE, sendProcessor, this.sendMessageExecutor);
        this.remotingServer.registerProcessor(RequestCode.SEND_MESSAGE_V2, sendProcessor, this.sendMessageExecutor);
        this.remotingServer.registerProcessor(RequestCode.SEND_BATCH_MESSAGE, sendProcessor, this.sendMessageExecutor);
        this.remotingServer.registerProcessor(RequestCode.CONSUMER_SEND_MSG_BACK, sendProcessor, this.sendMessageExecutor);
        this.fastRemotingServer.registerProcessor(RequestCode.SEND_MESSAGE, sendProcessor, this.sendMessageExecutor);
        this.fastRemotingServer.registerProcessor(RequestCode.SEND_MESSAGE_V2, sendProcessor, this.sendMessageExecutor);
        this.fastRemotingServer.registerProcessor(RequestCode.SEND_BATCH_MESSAGE, sendProcessor, this.sendMessageExecutor);
        this.fastRemotingServer.registerProcessor(RequestCode.CONSUMER_SEND_MSG_BACK, sendProcessor, this.sendMessageExecutor);
        /**
         * PullMessageProcessor
         */
        this.remotingServer.registerProcessor(RequestCode.PULL_MESSAGE, this.pullMessageProcessor, this.pullMessageExecutor);
        this.pullMessageProcessor.registerConsumeMessageHook(consumeMessageHookList);

所以關鍵的業務處理，後面只要去processorTable查找類型的對應Processor就行。

RocketMQ的同步結果推送與異步結果推送
RocketMQ的RemotingServer服務端，會維護一個responseTable，這是一個線程同步的Map結構。 key爲請求的ID，value是異步的消息結果。ConcurrentMap<Integer , ResponseFuture> 。

處理同步請求(NettyRemotingAbstract#invokeSyncImpl)時，處理的結果會存入responseTable，通過ResponseFuture提供一定的服務端異步處理支持，提升服務端的吞吐量。請求返回後，立即從responseTable中移除請求記錄。

處理異步請求(NettyRemotingAbstract#invokeAsyncImpl)時，處理的結果依然會存入responsTable，等待客戶端後續再來請求結果。但是他保存的依然是一個ResponseFuture，也就是在客戶端請求結果時再去獲取真正的結果。另外，在RemotingServer啓動時，會啓動一個定時的線程任務，不斷掃描responseTable，將其中過期的response清除掉。

1.4 Broker心跳註冊過程

BrokerController#start

        //K2 Broker核心的心跳註冊任務，需要深入解讀下。
        this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {

            @Override
            public void run() {
                try {
                    BrokerController.this.registerBrokerAll(true, false, brokerConfig.isForceRegister());
                } catch (Throwable e) {
                    log.error("registerBrokerAll Exception", e);
                }
            }
        }, 1000 * 10, Math.max(10000, Math.min(brokerConfig.getRegisterNameServerPeriod(), 60000)), TimeUnit.MILLISECONDS);

    public synchronized void registerBrokerAll(final boolean checkOrderConfig, boolean oneway, boolean forceRegister) {
        ...
        //這裏纔是比較關鍵的地方。先判斷是否需要註冊，然後調用doRegisterBrokerAll方法真正去註冊。
        if (forceRegister || needRegister(this.brokerConfig.getBrokerClusterName(),
            this.getBrokerAddr(),
            this.brokerConfig.getBrokerName(),
            this.brokerConfig.getBrokerId(),
            this.brokerConfig.getRegisterBrokerTimeoutMills())) {
            doRegisterBrokerAll(checkOrderConfig, oneway, topicConfigWrapper);
        }
    }

    //K2 Broker註冊最核心的部分
    private void doRegisterBrokerAll(boolean checkOrderConfig, boolean oneway,
        TopicConfigSerializeWrapper topicConfigWrapper) {
        List<RegisterBrokerResult> registerBrokerResultList = this.brokerOuterAPI.registerBrokerAll(
            this.brokerConfig.getBrokerClusterName(),
            this.getBrokerAddr(),
            this.brokerConfig.getBrokerName(),
            this.brokerConfig.getBrokerId(),
            this.getHAServerAddr(),
            topicConfigWrapper,
            this.filterServerManager.buildNewFilterServerList(),
            oneway,
            this.brokerConfig.getRegisterBrokerTimeoutMills(),
            this.brokerConfig.isCompressedRegister());

        if (registerBrokerResultList.size() > 0) {
            RegisterBrokerResult registerBrokerResult = registerBrokerResultList.get(0);
            if (registerBrokerResult != null) {
                //註冊完保存主從節點的地址
                if (this.updateMasterHAServerAddrPeriodically && registerBrokerResult.getHaServerAddr() != null) {
                    this.messageStore.updateHaMasterAddress(registerBrokerResult.getHaServerAddr());
                }

                this.slaveSynchronize.setMasterAddr(registerBrokerResult.getMasterAddr());

                if (checkOrderConfig) {
                    this.getTopicConfigManager().updateOrderTopicConfig(registerBrokerResult.getKvTable());
                }
            }
        }
    }

會向所有NameServer進行註冊：

    public List<RegisterBrokerResult> registerBrokerAll(
        final String clusterName,
        final String brokerAddr,
        final String brokerName,
        final long brokerId,
        final String haServerAddr,
        final TopicConfigSerializeWrapper topicConfigWrapper,
        final List<String> filterServerList,
        final boolean oneway,
        final int timeoutMills,
        final boolean compressed) {
        //使用CopyOnWriteArrayList提升併發安全性
        final List<RegisterBrokerResult> registerBrokerResultList = new CopyOnWriteArrayList<>();
        List<String> nameServerAddressList = this.remotingClient.getNameServerAddressList();
        if (nameServerAddressList != null && nameServerAddressList.size() > 0) {

            final RegisterBrokerRequestHeader requestHeader = new RegisterBrokerRequestHeader();
            requestHeader.setBrokerAddr(brokerAddr);
            requestHeader.setBrokerId(brokerId);
            requestHeader.setBrokerName(brokerName);
            requestHeader.setClusterName(clusterName);
            requestHeader.setHaServerAddr(haServerAddr);
            requestHeader.setCompressed(compressed);

            RegisterBrokerBody requestBody = new RegisterBrokerBody();
            requestBody.setTopicConfigSerializeWrapper(topicConfigWrapper);
            requestBody.setFilterServerList(filterServerList);
            final byte[] body = requestBody.encode(compressed);
            final int bodyCrc32 = UtilAll.crc32(body);
            requestHeader.setBodyCrc32(bodyCrc32);
            //通過CountDownLatch，保證在所有NameServer上完成註冊後再一起結束。
            final CountDownLatch countDownLatch = new CountDownLatch(nameServerAddressList.size());
            for (final String namesrvAddr : nameServerAddressList) {
                brokerOuterExecutor.execute(new Runnable() {
                    @Override
                    public void run() {
                        try {
                            RegisterBrokerResult result = registerBroker(namesrvAddr, oneway, timeoutMills, requestHeader, body);
                            if (result != null) {
                                registerBrokerResultList.add(result);
                            }

                            log.info("register broker[{}]to name server {} OK", brokerId, namesrvAddr);
                        } catch (Exception e) {
                            log.warn("registerBroker Exception, {}", namesrvAddr, e);
                        } finally {
                            countDownLatch.countDown();
                        }
                    }
                });
            }

            try {
                countDownLatch.await(timeoutMills, TimeUnit.MILLISECONDS);
            } catch (InterruptedException e) {
            }
        }

        return registerBrokerResultList;
    }

NameServer端的處理：
DefaultRequestProcessor#processRequest

    //K2 NameServer處理請求的核心代碼
    @Override
    public RemotingCommand processRequest(ChannelHandlerContext ctx,
        RemotingCommand request) throws RemotingCommandException {

        if (ctx != null) {
            log.debug("receive request, {} {} {}",
                request.getCode(),
                RemotingHelper.parseChannelRemoteAddr(ctx.channel()),
                request);
        }


        switch (request.getCode()) {
            case RequestCode.PUT_KV_CONFIG:
                return this.putKVConfig(ctx, request);
            case RequestCode.GET_KV_CONFIG:
                return this.getKVConfig(ctx, request);
            case RequestCode.DELETE_KV_CONFIG:
                return this.deleteKVConfig(ctx, request);
            case RequestCode.QUERY_DATA_VERSION:
                return queryBrokerTopicConfig(ctx, request);
            case RequestCode.REGISTER_BROKER: //Broker註冊請求處理。版本默認是當前框架版本
                Version brokerVersion = MQVersion.value2Version(request.getVersion());
                if (brokerVersion.ordinal() >= MQVersion.Version.V3_0_11.ordinal()) {
                    return this.registerBrokerWithFilterServer(ctx, request); //當前版本
                } else {
                    return this.registerBroker(ctx, request);
                }

最終會調用RouteInfoManager對Broker信息進行註冊：

    //K2 NameServer 實際處理Broker註冊的地方
    public RemotingCommand registerBrokerWithFilterServer(ChannelHandlerContext ctx, RemotingCommand request)
        throws RemotingCommandException {
        final RemotingCommand response = RemotingCommand.createResponseCommand(RegisterBrokerResponseHeader.class);
        final RegisterBrokerResponseHeader responseHeader = (RegisterBrokerResponseHeader) response.readCustomHeader();
        final RegisterBrokerRequestHeader requestHeader =
            (RegisterBrokerRequestHeader) request.decodeCommandCustomHeader(RegisterBrokerRequestHeader.class);

        if (!checksum(ctx, request, requestHeader)) {
            response.setCode(ResponseCode.SYSTEM_ERROR);
            response.setRemark("crc32 not match");
            return response;
        }

        RegisterBrokerBody registerBrokerBody = new RegisterBrokerBody();

        if (request.getBody() != null) {
            try {
                registerBrokerBody = RegisterBrokerBody.decode(request.getBody(), requestHeader.isCompressed());
            } catch (Exception e) {
                throw new RemotingCommandException("Failed to decode RegisterBrokerBody", e);
            }
        } else {
            registerBrokerBody.getTopicConfigSerializeWrapper().getDataVersion().setCounter(new AtomicLong(0));
            registerBrokerBody.getTopicConfigSerializeWrapper().getDataVersion().setTimestamp(0);
        }
        //routeInfoManager就是管理路由信息的核心組件。
        RegisterBrokerResult result = this.namesrvController.getRouteInfoManager().registerBroker(
            requestHeader.getClusterName(),
            requestHeader.getBrokerAddr(),
            requestHeader.getBrokerName(),
            requestHeader.getBrokerId(),
            requestHeader.getHaServerAddr(),
            registerBrokerBody.getTopicConfigSerializeWrapper(),
            registerBrokerBody.getFilterServerList(),
            ctx.channel());

        responseHeader.setHaServerAddr(result.getHaServerAddr());
        responseHeader.setMasterAddr(result.getMasterAddr());

        byte[] jsonValue = this.namesrvController.getKvConfigManager().getKVListByNamespace(NamesrvUtil.NAMESPACE_ORDER_TOPIC_CONFIG);
        response.setBody(jsonValue);

        response.setCode(ResponseCode.SUCCESS);
        response.setRemark(null);
        return response;
    }

看看RouteInfoManager 管理的路由信息：

public class RouteInfoManager {
    private static final InternalLogger log = InternalLoggerFactory.getLogger(LoggerName.NAMESRV_LOGGER_NAME);
    private final static long BROKER_CHANNEL_EXPIRED_TIME = 1000 * 60 * 2;
    private final ReadWriteLock lock = new ReentrantReadWriteLock();
    //幾個關鍵的Table
    private final HashMap<String/* topic */, List<QueueData>> topicQueueTable;
    private final HashMap<String/* brokerName */, BrokerData> brokerAddrTable;
    private final HashMap<String/* clusterName */, Set<String/* brokerName */>> clusterAddrTable;
    private final HashMap<String/* brokerAddr */, BrokerLiveInfo> brokerLiveTable;
    private final HashMap<String/* brokerAddr */, List<String>/* Filter Server */> filterServerTable;

2.發送消息

2.1 普通消息發送DefaultMQProducer

DefaultMQProducer#start

    public void start() throws MQClientException {
        this.setProducerGroup(withNamespace(this.producerGroup));
        this.defaultMQProducerImpl.start();
        if (null != traceDispatcher) {
            try {
                traceDispatcher.start(this.getNamesrvAddr(), this.getAccessChannel());
            } catch (MQClientException e) {
                log.warn("trace dispatcher start failed ", e);
            }
        }
    }

DefaultMQProducerImpl#start()

    public void start() throws MQClientException {
        this.start(true);
    }
    //K2 消息生產者的啓動方法
    public void start(final boolean startFactory) throws MQClientException {
        switch (this.serviceState) {
            case CREATE_JUST:
                this.serviceState = ServiceState.START_FAILED;

                this.checkConfig();
                //修改當前的instanceName爲當前進程ID
                if (!this.defaultMQProducer.getProducerGroup().equals(MixAll.CLIENT_INNER_PRODUCER_GROUP)) {
                    this.defaultMQProducer.changeInstanceNameToPID();
                }
                //客戶端核心的MQ客戶端工廠 對於事務消息發送者，在這裏面會完成事務消息的發送者的服務註冊
                this.mQClientFactory = MQClientManager.getInstance().getOrCreateMQClientInstance(this.defaultMQProducer, rpcHook);
                //註冊MQ客戶端工廠示例
                boolean registerOK = mQClientFactory.registerProducer(this.defaultMQProducer.getProducerGroup(), this);
                if (!registerOK) {
                    this.serviceState = ServiceState.CREATE_JUST;
                    throw new MQClientException("The producer group[" + this.defaultMQProducer.getProducerGroup()
                        + "] has been created before, specify another name please." + FAQUrl.suggestTodo(FAQUrl.GROUP_NAME_DUPLICATE_URL),
                        null);
                }

                this.topicPublishInfoTable.put(this.defaultMQProducer.getCreateTopicKey(), new TopicPublishInfo());
                //啓動示例 --所有客戶端組件都交由mQClientFactory啓動
                if (startFactory) {
                    mQClientFactory.start();
                }

                log.info("the producer [{}] start OK. sendMessageWithVIPChannel={}", this.defaultMQProducer.getProducerGroup(),
                    this.defaultMQProducer.isSendMessageWithVIPChannel());
                this.serviceState = ServiceState.RUNNING;
                break;
            case RUNNING:
            case START_FAILED:
            case SHUTDOWN_ALREADY:
                throw new MQClientException("The producer service state not OK, maybe started once, "
                    + this.serviceState
                    + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
                    null);
            default:
                break;
        }

        this.mQClientFactory.sendHeartbeatToAllBrokerWithLock();

        this.startScheduledTask();

    }

總結：

1）RocketMQ的所有客戶端實例，包括生產者和消費者，都是統一交由mQClientFactory組件來啓動，也就是說，所有客戶端的啓動流程是固定的，不同客戶端的區別只是在於他們在啓動前註冊的一些信息不同。例如生產者註冊到producerTable，消費者註冊到consumerTable，管理控制端註冊到adminExtTable
2）MQClientInstance#start啓動了很多服務
this.mQClientAPIImpl.fetchNameServerAddr();
this.mQClientAPIImpl.start();
this.startScheduledTask();
this.pullMessageService.start();
this.rebalanceService.start();
this.defaultMQProducer.getDefaultMQProducerImpl().start(false);

DefaultMQProducer#send(Message)
-> DefaultMQProducerImpl#send(Message)
-> DefaultMQProducerImpl#sendDefaultImpl

1）TopicPublishInfo topicPublishInfo = this.tryToFindTopicPublishInfo(msg.getTopic());獲取Topic信息
2）MessageQueue mqSelected = this.selectOneMessageQueue(topicPublishInfo, lastBrokerName);Producer根據發送者負載均衡策略，計算把消息發到哪個MessageQueue中（Producer在獲取路由信息後，會選出一個MessageQueue去發送消息。這個選MessageQueue的方法就是一個索引自增然後取模的方式。.sendLatencyFaultEnable默認是關閉的，Broker故障延遲機制，表示一種發送消息失敗後一定時間內不再往同一個Queue重複發送的機制）
3）sendResult = this.sendKernelImpl(msg, mq, communicationMode, sendCallback, topicPublishInfo, timeout - costTime);實際發送消息的方法（根據MessageQueue 獲取對應的Broker地址）

Producer如何管理Borker路由信息？

Producer需要拉取Broker列表，然後跟Broker建立連接等等很多核心的流程，其實都是在發送消息時建立的。Send方法中，首先需要獲得Topic的路由信息。這會從本地緩存中獲取，如果本地緩存中沒有，就從NameServer中去申請。

    //找路由表的過程都是先從本地緩存找，本地緩存沒有，就去NameServer上申請
    private TopicPublishInfo tryToFindTopicPublishInfo(final String topic) {
        TopicPublishInfo topicPublishInfo = this.topicPublishInfoTable.get(topic);
        if (null == topicPublishInfo || !topicPublishInfo.ok()) {
            this.topicPublishInfoTable.putIfAbsent(topic, new TopicPublishInfo());
            //Producer向NameServer獲取更新Topic的路由信息
            this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic);
            //還是從本地緩存中尋找Topic的路由信息
            topicPublishInfo = this.topicPublishInfoTable.get(topic);
        }

        if (topicPublishInfo.isHaveTopicRouterInfo() || topicPublishInfo.ok()) {
            return topicPublishInfo;
        } else {
            this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic, true, this.defaultMQProducer);
            topicPublishInfo = this.topicPublishInfoTable.get(topic);
            return topicPublishInfo;
        }
    }

2.2 事務消息TransactionMQProducer

    //事務消息的啓動過程。啓動過程中會完成Processor的註冊
    @Override
    public void start() throws MQClientException {
        this.defaultMQProducerImpl.initTransactionEnv();
        super.start();
    }

    public void initTransactionEnv() {
        TransactionMQProducer producer = (TransactionMQProducer) this.defaultMQProducer;
        if (producer.getExecutorService() != null) {
            this.checkExecutor = producer.getExecutorService();
        } else {
            this.checkRequestQueue = new LinkedBlockingQueue<Runnable>(producer.getCheckRequestHoldMax());
            this.checkExecutor = new ThreadPoolExecutor(
                producer.getCheckThreadPoolMinSize(),
                producer.getCheckThreadPoolMaxSize(),
                1000 * 60,
                TimeUnit.MILLISECONDS,
                this.checkRequestQueue);
        }
    }

這裏唯一區別就是多了一個線程池checkExecutor。

    public TransactionSendResult sendMessageInTransaction(final Message msg,
        final Object arg) throws MQClientException {
        if (null == this.transactionListener) {
            throw new MQClientException("TransactionListener is null", null);
        }

        msg.setTopic(NamespaceUtil.wrapNamespace(this.getNamespace(), msg.getTopic()));
        return this.defaultMQProducerImpl.sendMessageInTransaction(msg, null, arg);
    }

DefaultMQProducerImpl#sendMessageInTransaction

1）TransactionListener transactionListener = getCheckListener();獲取監聽器
2）設置消息屬性
MessageAccessor.putProperty(msg, MessageConst.PROPERTY_TRANSACTION_PREPARED, "true");
MessageAccessor.putProperty(msg, MessageConst.PROPERTY_PRODUCER_GROUP, this.defaultMQProducer.getProducerGroup());
3）發送消息sendResult = this.send(msg);
4）發送成功，執行本地事務localTransactionState = transactionListener.executeLocalTransaction(msg, arg);
5）發送失敗（各種錯誤原因），設置爲回滾狀態
6）根據狀態進行處理this.endTransaction(msg, sendResult, localTransactionState, localException);這裏會向broker發起提交或者回滾請求

客戶端處理Broker回查請求：
ClientRemotingProcessor#processRequest
-> RequestCode.CHECK_TRANSACTION_STATE
ClientRemotingProcessor#checkTransactionState
-> DefaultMQProducerImpl#checkTransactionState
-> this.checkExecutor.submit(request);
-> localTransactionState = transactionListener.checkLocalTransaction(message);

3.消費消息

DefaultMQPushConsumer#start

    public void start() throws MQClientException {
        setConsumerGroup(NamespaceUtil.wrapNamespace(this.getNamespace(), this.consumerGroup));
        this.defaultMQPushConsumerImpl.start();
        if (null != traceDispatcher) {
            try {
                traceDispatcher.start(this.getNamesrvAddr(), this.getAccessChannel());
            } catch (MQClientException e) {
                log.warn("trace dispatcher start failed ", e);
            }
        }
    }

DefaultMQPushConsumerImpl#start

1）CLUSTERING模式，this.defaultMQPushConsumer.changeInstanceNameToPID();
2）this.mQClientFactory = MQClientManager.getInstance().getOrCreateMQClientInstance(this.defaultMQPushConsumer, this.rpcHook);客戶端示例工廠，生產者也是交由這個工廠啓動的。
3）this.rebalanceImpl.setAllocateMessageQueueStrategy(
this.defaultMQPushConsumer.getAllocateMessageQueueStrategy());負載均衡策略
4）this.pullAPIWrapper = new PullAPIWrapper();
5）廣播模式與集羣模式的最本質區別就是offset存儲的地方不一樣。
廣播模式是在消費者本地存儲offset：this.offsetStore = new LocalFileOffsetStore();
集羣模式是在Broker遠端存儲offset：this.offsetStore = new RemoteBrokerOffsetStore();
6）消費者服務
順序消費監聽創建ConsumeMessageOrderlyService；
併發消費監聽創建ConsumeMessageConcurrentlyService；
this.consumeMessageService.start();
7）註冊消費者。與生產者類似，客戶端只要按要求註冊即可，後續會隨mQClientFactory一起啓動。
mQClientFactory.registerConsumer(this.defaultMQPushConsumer.getConsumerGroup(), this);
8） mQClientFactory.start();
9）this.updateTopicSubscribeInfoWhenSubscriptionChanged();
10）this.mQClientFactory.checkClientInBroker();
11）this.mQClientFactory.sendHeartbeatToAllBrokerWithLock();
12）this.mQClientFactory.rebalanceImmediately();

消費端負載均衡：

AllocateMachineRoomNearby：將同機房的Consumer和Broker優先分配在一起。
AllocateMessageQueueAveragely：平均分配。將所有MessageQueue平均分給每一個消費者
AllocateMessageQueueAveragelyByCircle：輪詢分配。輪流的給一個消費者分配一個MessageQueue。
AllocateMessageQueueByConfig：不分配，直接指定一個messageQueue列表。類似於廣播模式，直接指定所有隊列。
AllocateMessageQueueByMachineRoom：按邏輯機房的概念進行分配。又是對BrokerName和ConsumerIdc有定製化的配置。
AllocateMessageQueueConsistentHash。源碼中有測試代碼AllocateMessageQueueConsitentHashTest。這個一致性哈希策略只需要指定一個虛擬節點數，是用的一個哈希環的算法，虛擬節點是爲了讓Hash數據在換上分佈更爲均勻。

消費者服務的串聯：

DefaultMQPushConsumerImpl#start
this.mQClientFactory = MQClientManager.getInstance().getOrCreateMQClientInstance(this.defaultMQPushConsumer, this.rpcHook);
instance = new MQClientInstance()
this.pullMessageService = new PullMessageService(this);
PullMessageService#run
PullMessageService#pullMessage
DefaultMQPushConsumerImpl#pullMessage
DefaultMQPushConsumerImpl.this.consumeMessageService.submitConsumeRequest()消費者消息服務處理消費到的消息
ConsumeMessageConcurrentlyService#submitConsumeRequest
ConsumeMessageOrderlyService#submitConsumeRequest
this.consumeExecutor.submit(consumeRequest);併發和順序消費的線程池線程數都爲20
ConsumeMessageConcurrentlyService.ConsumeRequest#run
ConsumeMessageOrderlyService.ConsumeRequest#run 需要加鎖
status = listener.consumeMessage(Collections.unmodifiableList(msgs), context);這個listener就是自定義的業務邏輯。

4.Broker端

4.1 文件存儲

DefaultMessageStore#putMessage

    //K1 Broker典型的消息存儲處理
    //當前版本將默認的寫入方式更改成了異步寫入機制。
    @Override
    public PutMessageResult putMessage(MessageExtBrokerInner msg) {
        try {
            return asyncPutMessage(msg).get();
        } catch (InterruptedException | ExecutionException e) {
            return new PutMessageResult(PutMessageStatus.UNKNOWN_ERROR, null);
        }
    }

DefaultMessageStore#asyncPutMessage
-> CommitLog#asyncPutMessage

1）遲消息的實現方式，就要修改一下msg的topic和queueID，改爲系統默認創建的延遲隊列。topic是固定的SCHEDULE_TOPIC_XXXX，queueId是根據延遲級別選擇的。
2）加鎖putMessageLock.lock();
3）找到最後一個CommitLog文件。最後一個就是當前寫的文件
4）mappedFile.appendMessage(msg, this.appendMessageCallback, putMessageContext); 以零拷貝的方式實現消息順序寫。 ByteBuffer.allocateDirect(fileSize)
5）putMessageLock.unlock();
6）submitFlushRequest(result, msg);提交刷盤請求
同步刷盤機制：向this.flushCommitLogService提交GroupCommitRequest請求，每10ms執行一次flush；
異步刷盤機制：flushCommitLogService.wakeup();或者commitLogService.wakeup();
7）submitReplicaRequest(result, msg);提交主從同步請求。
-> HAService#putRequest
-> this.groupTransferService.putRequest(request);
HAService有三個核心組件，與Master相關的是acceptSocketService和groupTransferService。其中acceptSocketService主要負責維護Master與Slave之間的TCP連接。groupTransferService主要與主從同步複製有關。而slave相關的則是haClient。

接下來看看ConsumeQueue和IndexFile的寫入：

DefaultMessageStore#start
this.reputMessageService.start();
DefaultMessageStore.ReputMessageService#run，Commit日誌分發服務，每隔1毫秒，會檢查是否需要（就是看有沒有新數據）向ConsumeQueue和IndexFile中轉發一次CommitLog寫入的消息。
DefaultMessageStore.ReputMessageService#doReput
DefaultMessageStore.this.doDispatch(dispatchRequest);
CommitLogDispatcherBuildConsumeQueue#doDispatch
CommitLogDispatcherBuildIndex#doDispatch
長輪詢：如果有消息到了主節點，並且開啓了長輪詢。就要通過長輪詢機制通知消費者，新消息已經到了，可以消費了。DefaultMessageStore.this.messageArrivingListener.arriving()，實例是NotifyMessageArrivingListener

定時刪除過期文件：

DefaultMessageStore#start
DefaultMessageStore#addScheduleTask
定時任務：DefaultMessageStore.this.cleanFilesPeriodically();
this.cleanCommitLogService.run();定時刪除過期commitlog
this.cleanConsumeQueueService.run();定時刪除過期的consumequeue

4.2 長輪詢機制

RocketMQ對消息消費者提供了Push推模式和Pull拉模式兩種消費模式。但是這兩種消費模式的本質其實都是Pull拉模式，Push模式可以認爲是一種定時的Pull機制。但是這時有一個問題，當使用Push模式時，如果RocketMQ中沒有對應的數據，那難道一直進行空輪詢嗎？如果是這樣的話，那顯然會極大的浪費網絡帶寬以及服務器的性能，並且，當有新的消息進來時，RocketMQ也沒有辦法儘快通知客戶端，而只能等客戶端下一次來拉取消息了。針對這個問題，RocketMQ實現了一種長輪詢機制 long polling。

長輪詢機制簡單來說，就是當Broker接收到Consumer的Pull請求時，判斷如果沒有對應的消息，不用直接給Consumer響應(給響應也是個空的，沒意義)，而是就將這個Pull請求給緩存起來。當Producer發送消息過來時，增加一個步驟去檢查是否有對應的已緩存的Pull請求，如果有，就及時將請求從緩存中拉取出來，並將消息通知給Consumer。

消費者拉取消息：
BrokerController#registerProcessor
-> this.remotingServer.registerProcessor(RequestCode.PULL_MESSAGE, this.pullMessageProcessor, this.pullMessageExecutor);
-> PullMessageProcessor#processRequest()
-> ResponseCode.PULL_NOT_FOUND，消息長輪詢1：消費者消費時，沒有消息就會被緩存起來。brokerAllowSuspend 客戶端初次請求消息時是指定的true。重新喚醒時指定爲false，hasSuspendFlag默認都是true。
this.brokerController.getPullRequestHoldService().suspendPullRequest( topic, queueId, pullRequest);請求是PullRequest

生產者發送消息：
BrokerController#registerProcessor

    public void registerProcessor() {
        /**
         * SendMessageProcessor
         */
        SendMessageProcessor sendProcessor = new SendMessageProcessor(this);
        sendProcessor.registerSendMessageHook(sendMessageHookList);
        sendProcessor.registerConsumeMessageHook(consumeMessageHookList);

        this.remotingServer.registerProcessor(RequestCode.SEND_MESSAGE, sendProcessor, this.sendMessageExecutor);
        this.remotingServer.registerProcessor(RequestCode.SEND_MESSAGE_V2, sendProcessor, this.sendMessageExecutor);
        this.remotingServer.registerProcessor(RequestCode.SEND_BATCH_MESSAGE, sendProcessor, this.sendMessageExecutor);
        this.remotingServer.registerProcessor(RequestCode.CONSUMER_SEND_MSG_BACK, sendProcessor, this.sendMessageExecutor);
        this.fastRemotingServer.registerProcessor(RequestCode.SEND_MESSAGE, sendProcessor, this.sendMessageExecutor);
        this.fastRemotingServer.registerProcessor(RequestCode.SEND_MESSAGE_V2, sendProcessor, this.sendMessageExecutor);
        this.fastRemotingServer.registerProcessor(RequestCode.SEND_BATCH_MESSAGE, sendProcessor, this.sendMessageExecutor);
        this.fastRemotingServer.registerProcessor(RequestCode.CONSUMER_SEND_MSG_BACK, sendProcessor, this.sendMessageExecutor);
        /**
         * PullMessageProcessor
         */
        this.remotingServer.registerProcessor(RequestCode.PULL_MESSAGE, this.pullMessageProcessor, this.pullMessageExecutor);
        this.pullMessageProcessor.registerConsumeMessageHook(consumeMessageHookList);

SendMessageProcessor#processRequest

SendMessageProcessor#asyncProcessRequest()
SendMessageProcessor#asyncSendMessage
DefaultMessageStore#asyncPutMessage（參見4.1）

接下來看看關聯的ReputMessageService

DefaultMessageStore#start
this.reputMessageService.start();
DefaultMessageStore.ReputMessageService#run，Commit日誌分發服務，每隔1毫秒，會檢查是否需要（就是看有沒有新數據）向ConsumeQueue和IndexFile中轉發一次CommitLog寫入的消息。
DefaultMessageStore.ReputMessageService#doReput
DefaultMessageStore.this.doDispatch(dispatchRequest);
CommitLogDispatcherBuildConsumeQueue#doDispatch
CommitLogDispatcherBuildIndex#doDispatch
長輪詢：如果有消息到了主節點，並且開啓了長輪詢。就要通過長輪詢機制通知消費者，新消息已經到了，可以消費了。DefaultMessageStore.this.messageArrivingListener.arriving()，實例是NotifyMessageArrivingListener
NotifyMessageArrivingListener#arriving 長輪詢：生產者發送消息後的監聽事件
this.pullRequestHoldService.notifyMessageArriving()
PullRequestHoldService#notifyMessageArriving() 會去this.pullRequestTable.get(key);查找

RocketMQ源碼剖析 1.各個組件啓動源碼、框架結構 2.發送消息 3.消費消息 4.Broker端

1.各個組件啓動源碼、框架結構

1.1 NameServer啓動

1.2 Broker啓動

1.3 Netty網絡框架

1.4 Broker心跳註冊過程

2.發送消息

2.1 普通消息發送DefaultMQProducer

2.2 事務消息TransactionMQProducer

3.消費消息

4.Broker端

4.1 文件存儲

4.2 長輪詢機制

[轉帖]使用NMT和pmap解決JVM資源泄漏問題原創

Python實現大麥網搶票的四大關鍵技術點解析

salesforce零基礎學習（一百三十八）零碎知識點小總結（十）

一款開源的.NET程序集反編譯、編輯和調試神器

關於接口協議，你必須要知道這些！

【2024-05-21】以茶會友

webflux項目的兩個配置問題

Project Reactor源碼分析2-線程切換 1.聲明階段 2.subscribe、onSubscribe、request階段 3.總結參考

Project Reactor源碼分析1-底層實現原理分析 1.聲明階段 2.subscribe 參考

DDD筆記1：簡介及戰略設計 1.DDD的背景和來源 2.戰略設計參考

DDD筆記5：實踐中的問題和關鍵點 1.CI/CD 2.領域溝通和建模如何避免漏掉重要細節 3.DDD的常見問題、爭論以及侷限性參考

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結