RocketMQ中PullConsumer的啓動源碼分析

通過DefaultMQPullConsumer作爲默認實現，這裏的啓動過程和Producer很相似，但相比複雜一些
【RocketMQ中Producer的啓動源碼分析】

DefaultMQPullConsumer的構造方法：

public DefaultMQPullConsumer(final String consumerGroup, RPCHook rpcHook) {
    this.consumerGroup = consumerGroup;
    defaultMQPullConsumerImpl = new DefaultMQPullConsumerImpl(this, rpcHook);
}

這裏會封裝一個DefaultMQPullConsumerImpl，類似於Producer中DefaultMQProducerImpl

DefaultMQPullConsumerImpl：

public class DefaultMQPullConsumerImpl implements MQConsumerInner {
    private final InternalLogger log = ClientLogger.getLog();
    private final DefaultMQPullConsumer defaultMQPullConsumer;
    private final long consumerStartTimestamp = System.currentTimeMillis();
    private final RPCHook rpcHook;
    private final ArrayList<ConsumeMessageHook> consumeMessageHookList = new ArrayList<ConsumeMessageHook>();
    private final ArrayList<FilterMessageHook> filterMessageHookList = new ArrayList<FilterMessageHook>();
    private volatile ServiceState serviceState = ServiceState.CREATE_JUST;
    private MQClientInstance mQClientFactory;
    private PullAPIWrapper pullAPIWrapper;
    private OffsetStore offsetStore;
    private RebalanceImpl rebalanceImpl = new RebalancePullImpl(this);

    public DefaultMQPullConsumerImpl(final DefaultMQPullConsumer defaultMQPullConsumer, final RPCHook rpcHook) {
        this.defaultMQPullConsumer = defaultMQPullConsumer;
        this.rpcHook = rpcHook;
    }
    ......
}

如上會封裝這些東西，在後面遇到了再詳細介紹

而DefaultMQPullConsumer的start方法，其實際上調用的是DefaultMQPullConsumerImpl的start方法

DefaultMQPullConsumerImpl的start方法：

public synchronized void start() throws MQClientException {
    switch (this.serviceState) {
        case CREATE_JUST:
            this.serviceState = ServiceState.START_FAILED;

            this.checkConfig();

            this.copySubscription();

            if (this.defaultMQPullConsumer.getMessageModel() == MessageModel.CLUSTERING) {
                this.defaultMQPullConsumer.changeInstanceNameToPID();
            }

            this.mQClientFactory = MQClientManager.getInstance().getAndCreateMQClientInstance(this.defaultMQPullConsumer, this.rpcHook);

            this.rebalanceImpl.setConsumerGroup(this.defaultMQPullConsumer.getConsumerGroup());
            this.rebalanceImpl.setMessageModel(this.defaultMQPullConsumer.getMessageModel());
            this.rebalanceImpl.setAllocateMessageQueueStrategy(this.defaultMQPullConsumer.getAllocateMessageQueueStrategy());
            this.rebalanceImpl.setmQClientFactory(this.mQClientFactory);

            this.pullAPIWrapper = new PullAPIWrapper(
                mQClientFactory,
                this.defaultMQPullConsumer.getConsumerGroup(), isUnitMode());
            this.pullAPIWrapper.registerFilterMessageHook(filterMessageHookList);

            if (this.defaultMQPullConsumer.getOffsetStore() != null) {
                this.offsetStore = this.defaultMQPullConsumer.getOffsetStore();
            } else {
                switch (this.defaultMQPullConsumer.getMessageModel()) {
                    case BROADCASTING:
                        this.offsetStore = new LocalFileOffsetStore(this.mQClientFactory, this.defaultMQPullConsumer.getConsumerGroup());
                        break;
                    case CLUSTERING:
                        this.offsetStore = new RemoteBrokerOffsetStore(this.mQClientFactory, this.defaultMQPullConsumer.getConsumerGroup());
                        break;
                    default:
                        break;
                }
                this.defaultMQPullConsumer.setOffsetStore(this.offsetStore);
            }

            this.offsetStore.load();

            boolean registerOK = mQClientFactory.registerConsumer(this.defaultMQPullConsumer.getConsumerGroup(), this);
            if (!registerOK) {
                this.serviceState = ServiceState.CREATE_JUST;

                throw new MQClientException("The consumer group[" + this.defaultMQPullConsumer.getConsumerGroup()
                    + "] has been created before, specify another name please." + FAQUrl.suggestTodo(FAQUrl.GROUP_NAME_DUPLICATE_URL),
                    null);
            }

            mQClientFactory.start();
            log.info("the consumer [{}] start OK", this.defaultMQPullConsumer.getConsumerGroup());
            this.serviceState = ServiceState.RUNNING;
            break;
        case RUNNING:
        case START_FAILED:
        case SHUTDOWN_ALREADY:
            throw new MQClientException("The PullConsumer service state not OK, maybe started once, "
                + this.serviceState
                + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
                null);
        default:
            break;
    }

}

首先checkConfig方法會對配置做檢查

接着copySubscription方法：

private void copySubscription() throws MQClientException {
    try {
        Set<String> registerTopics = this.defaultMQPullConsumer.getRegisterTopics();
        if (registerTopics != null) {
            for (final String topic : registerTopics) {
                SubscriptionData subscriptionData = FilterAPI.buildSubscriptionData(this.defaultMQPullConsumer.getConsumerGroup(),
                    topic, SubscriptionData.SUB_ALL);
                this.rebalanceImpl.getSubscriptionInner().put(topic, subscriptionData);
            }
        }
    } catch (Exception e) {
        throw new MQClientException("subscription exception", e);
    }
}

這裏的registerTopics是由用戶調用setRegisterTopics方法註冊進來的Topic集合
在這裏會將集合中的Topic包裝成SubscriptionData保存在rebalanceImpl中

SubscriptionData：

public class SubscriptionData implements Comparable<SubscriptionData> {
    public final static String SUB_ALL = "*";
    private boolean classFilterMode = false;
    private String topic;
    private String subString;
    private Set<String> tagsSet = new HashSet<String>();
    private Set<Integer> codeSet = new HashSet<Integer>();
    private long subVersion = System.currentTimeMillis();
    private String expressionType = ExpressionType.TAG;
    ......
}

RebalanceImpl：

public abstract class RebalanceImpl {
    protected final ConcurrentMap<MessageQueue, ProcessQueue> processQueueTable = new ConcurrentHashMap<MessageQueue, ProcessQueue>(64);
    protected final ConcurrentMap<String/* topic */, Set<MessageQueue>> topicSubscribeInfoTable =
        new ConcurrentHashMap<String, Set<MessageQueue>>();
    protected final ConcurrentMap<String /* topic */, SubscriptionData> subscriptionInner =
        new ConcurrentHashMap<String, SubscriptionData>();
    protected String consumerGroup;
    protected MessageModel messageModel;
    protected AllocateMessageQueueStrategy allocateMessageQueueStrategy;
    protected MQClientInstance mQClientFactory;
	......
}

回到start方法，接着和Producer中一樣通過MQClientManager獲取一個MQClientInstance
然後會完成對rebalanceImpl屬性的填充

接着會實例化一個PullAPIWrapper，同時向其註冊過濾器的鉤子，這個對象在之後分析消息拉取時詳細介紹

接下來會根據消息的模式，決定使用不同方式的OffsetStore

public enum MessageModel {
    /**
     * broadcast
     */
    BROADCASTING("BROADCASTING"),
    /**
     * clustering
     */
    CLUSTERING("CLUSTERING");
	......
}

分別是廣播模式和集羣模式
廣播模式（BROADCASTING）：同一個ConsumerGroup裏的每個Consumer都能消費到所訂閱Topic的全部消息，也就是一個消息會被多次分發，被多個Consumer消費
集羣模式（CLUSTERING）：同一個ConsumerGroup裏的每個Consumer只消費所訂閱消息的一部分內容，同一個ConsumerGroup裏所有的Consumer消費的內容合起來纔是所訂閱Topic內容的整體

採用廣播模式，消費者的消費進度offset會被保存在本地；而採用集羣模式，消費者的消費進度offset會被保存在遠端（broker）上
故廣播模式使用LocalFileOffsetStore，集羣模式使用RemoteBrokerOffsetStore

在採用廣播模式，即LocalFileOffsetStore，調用load方法會對其配置文件offsets.json進行加載，而RemoteBrokerOffsetStore時沒意義的異步操作
LocalFileOffsetStore的load方法：

public void load() throws MQClientException {
    OffsetSerializeWrapper offsetSerializeWrapper = this.readLocalOffset();
    if (offsetSerializeWrapper != null && offsetSerializeWrapper.getOffsetTable() != null) {
        offsetTable.putAll(offsetSerializeWrapper.getOffsetTable());

        for (MessageQueue mq : offsetSerializeWrapper.getOffsetTable().keySet()) {
            AtomicLong offset = offsetSerializeWrapper.getOffsetTable().get(mq);
            log.info("load consumer's offset, {} {} {}",
                this.groupName,
                mq,
                offset.get());
        }
    }
}

readLocalOffset方法會將offsets.json文件中的json字符串轉換成OffsetSerializeWrapper對象封裝

public class OffsetSerializeWrapper extends RemotingSerializable {
    private ConcurrentMap<MessageQueue, AtomicLong> offsetTable =
        new ConcurrentHashMap<MessageQueue, AtomicLong>();

    public ConcurrentMap<MessageQueue, AtomicLong> getOffsetTable() {
        return offsetTable;
    }

    public void setOffsetTable(ConcurrentMap<MessageQueue, AtomicLong> offsetTable) {
        this.offsetTable = offsetTable;
    }
}

從這裏就可裏大致理解json文件中的內容，其中AtomicLong就對應MessageQueue下具體的Offset
之後在load方法中，會將該map保存在LocalFileOffsetStore中的offsetTable中

接着會調用mQClientFactory的start方法，這個方法在 【RocketMQ中Producer的啓動源碼分析】 中進行過分析

public void start() throws MQClientException {
    synchronized (this) {
        switch (this.serviceState) {
            case CREATE_JUST:
                this.serviceState = ServiceState.START_FAILED;
                // If not specified,looking address from name server
                if (null == this.clientConfig.getNamesrvAddr()) {
                    this.mQClientAPIImpl.fetchNameServerAddr();
                }
                // Start request-response channel
                this.mQClientAPIImpl.start();
                // Start various schedule tasks
                this.startScheduledTask();
                // Start pull service
                this.pullMessageService.start();
                // Start rebalance service
                this.rebalanceService.start();
                // Start push service
                this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
                log.info("the client factory [{}] start OK", this.clientId);
                this.serviceState = ServiceState.RUNNING;
                break;
            case RUNNING:
                break;
            case SHUTDOWN_ALREADY:
                break;
            case START_FAILED:
                throw new MQClientException("The Factory object[" + this.getClientId() + "] has been created before, and failed.", null);
            default:
                break;
        }
    }
}

首先若是沒有設置NameServer的地址，會調用fetchNameServerAddr方法進行自動尋址，詳見Producer的啓動

之後mQClientAPIImpl的start方法會完成對Netty客戶端的綁定操作，詳見Producer的啓動

startScheduledTask方法則會設置五個定時任務：
①若是名稱服務地址namesrvAddr不存在，則調用前面的fetchNameServerAddr方法，定時更新名稱服務
②定時更新Topic所對應的路由信息
③定時清除離線的Broker，以及向當前在線的Broker發送心跳包
（以上詳見Producer的啓動）

④定時持久化消費者隊列的消費進度
DefaultMQPullConsumerImpl中的實現：

public void persistConsumerOffset() {
    try {
        this.makeSureStateOK();
        Set<MessageQueue> mqs = new HashSet<MessageQueue>();
        Set<MessageQueue> allocateMq = this.rebalanceImpl.getProcessQueueTable().keySet();
        mqs.addAll(allocateMq);
        this.offsetStore.persistAll(mqs);
    } catch (Exception e) {
        log.error("group: " + this.defaultMQPullConsumer.getConsumerGroup() + " persistConsumerOffset exception", e);
    }
}

首先從rebalanceImpl中取出所有處理的消費隊列MessageQueue集合
然後調用offsetStore的persistAll方法進一步處理該集合

由於廣播模式和集羣模式，所以這裏有兩種實現：
廣播模式LocalFileOffsetStore的persistAll方法：

public void persistAll(Set<MessageQueue> mqs) {
    if (null == mqs || mqs.isEmpty())
        return;

    OffsetSerializeWrapper offsetSerializeWrapper = new OffsetSerializeWrapper();
    for (Map.Entry<MessageQueue, AtomicLong> entry : this.offsetTable.entrySet()) {
        if (mqs.contains(entry.getKey())) {
            AtomicLong offset = entry.getValue();
            offsetSerializeWrapper.getOffsetTable().put(entry.getKey(), offset);
        }
    }

    String jsonString = offsetSerializeWrapper.toJson(true);
    if (jsonString != null) {
        try {
            MixAll.string2File(jsonString, this.storePath);
        } catch (IOException e) {
            log.error("persistAll consumer offset Exception, " + this.storePath, e);
        }
    }
}

這裏和之前的load方法相反，會將MessageQueue對應的offset信息替換掉原來的json文件中的內容
這樣就完成了廣播模式下定時持久化消費者隊列的消費進度

集羣模式RemoteBrokerOffsetStore的persistAll方法的實現：

public void persistAll(Set<MessageQueue> mqs) {
    if (null == mqs || mqs.isEmpty())
        return;

    final HashSet<MessageQueue> unusedMQ = new HashSet<MessageQueue>();
    if (!mqs.isEmpty()) {
        for (Map.Entry<MessageQueue, AtomicLong> entry : this.offsetTable.entrySet()) {
            MessageQueue mq = entry.getKey();
            AtomicLong offset = entry.getValue();
            if (offset != null) {
                if (mqs.contains(mq)) {
                    try {
                        this.updateConsumeOffsetToBroker(mq, offset.get());
                        log.info("[persistAll] Group: {} ClientId: {} updateConsumeOffsetToBroker {} {}",
                            this.groupName,
                            this.mQClientFactory.getClientId(),
                            mq,
                            offset.get());
                    } catch (Exception e) {
                        log.error("updateConsumeOffsetToBroker exception, " + mq.toString(), e);
                    }
                } else {
                    unusedMQ.add(mq);
                }
            }
        }
    }

    if (!unusedMQ.isEmpty()) {
        for (MessageQueue mq : unusedMQ) {
            this.offsetTable.remove(mq);
            log.info("remove unused mq, {}, {}", mq, this.groupName);
        }
    }
}

和上面類似，遍歷offsetTable中的內容，只不過不是保存在了本地，而是通過updateConsumeOffsetToBroker向Broker發送
updateConsumeOffsetToBroker方法：

private void updateConsumeOffsetToBroker(MessageQueue mq, long offset) throws RemotingException,
    MQBrokerException, InterruptedException, MQClientException {
    updateConsumeOffsetToBroker(mq, offset, true);
}

public void updateConsumeOffsetToBroker(MessageQueue mq, long offset, boolean isOneway) throws RemotingException,
    MQBrokerException, InterruptedException, MQClientException {
    FindBrokerResult findBrokerResult = this.mQClientFactory.findBrokerAddressInAdmin(mq.getBrokerName());
    if (null == findBrokerResult) {

        this.mQClientFactory.updateTopicRouteInfoFromNameServer(mq.getTopic());
        findBrokerResult = this.mQClientFactory.findBrokerAddressInAdmin(mq.getBrokerName());
    }

    if (findBrokerResult != null) {
        UpdateConsumerOffsetRequestHeader requestHeader = new UpdateConsumerOffsetRequestHeader();
        requestHeader.setTopic(mq.getTopic());
        requestHeader.setConsumerGroup(this.groupName);
        requestHeader.setQueueId(mq.getQueueId());
        requestHeader.setCommitOffset(offset);

        if (isOneway) {
            this.mQClientFactory.getMQClientAPIImpl().updateConsumerOffsetOneway(
                findBrokerResult.getBrokerAddr(), requestHeader, 1000 * 5);
        } else {
            this.mQClientFactory.getMQClientAPIImpl().updateConsumerOffset(
                findBrokerResult.getBrokerAddr(), requestHeader, 1000 * 5);
        }
    } else {
        throw new MQClientException("The broker[" + mq.getBrokerName() + "] not exist", null);
    }
}

首先根據BrokerName查找Broker的路由信息：

public FindBrokerResult findBrokerAddressInAdmin(final String brokerName) {
    String brokerAddr = null;
    boolean slave = false;
    boolean found = false;

    HashMap<Long/* brokerId */, String/* address */> map = this.brokerAddrTable.get(brokerName);
    if (map != null && !map.isEmpty()) {
        for (Map.Entry<Long, String> entry : map.entrySet()) {
            Long id = entry.getKey();
            brokerAddr = entry.getValue();
            if (brokerAddr != null) {
                found = true;
                if (MixAll.MASTER_ID == id) {
                    slave = false;
                } else {
                    slave = true;
                }
                break;

            }
        } // end of for
    }

    if (found) {
        return new FindBrokerResult(brokerAddr, slave, findBrokerVersion(brokerName, brokerAddr));
    }

    return null;
}

brokerAddrTable中的borker的路由信息會由 ②定時更新Topic所對應的路由信息 ，來完成更新，在brokerAddrTable中只要找的一個Broker的信息後，將其封裝爲FindBrokerResult返回

若是沒有找到會執行updateTopicRouteInfoFromNameServer方法，也就是執行了一次定時任務中的方法，立即更新一次，再通過findBrokerAddressInAdmin方法，重新查找

找到之後，實例化一個請求頭 UpdateConsumerOffsetRequestHeader，將相應信息封裝，由於使用的是Oneway模式，所以這裏採用updateConsumerOffsetOneway方法，通過Netty向Broker發送

public void updateConsumerOffsetOneway(
    final String addr,
    final UpdateConsumerOffsetRequestHeader requestHeader,
    final long timeoutMillis
) throws RemotingConnectException, RemotingTooMuchRequestException, RemotingTimeoutException, RemotingSendRequestException,
    InterruptedException {
    RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.UPDATE_CONSUMER_OFFSET, requestHeader);

    this.remotingClient.invokeOneway(MixAll.brokerVIPChannel(this.clientConfig.isVipChannelEnabled(), addr), request, timeoutMillis);
}

其實這裏就非常簡單地調用了invokeOneway方法，完成向Broker的消息單向發送
【RocketMQ中Producer消息的發送源碼分析】
非OneWay則採用同步發送
這樣，在集羣模式下，消費進度也就交給了Broker管理，之後的負載均衡以此爲基礎

⑤定時調整消費者端的線程池的大小
這裏針對的是PushConsumer，後續博客再介紹

對於PullConsumer來說rebalanceService服務的開啓纔是最重要的

RebalanceService：

public void run() {
    log.info(this.getServiceName() + " service started");

    while (!this.isStopped()) {
        this.waitForRunning(waitInterval);
        this.mqClientFactory.doRebalance();
    }

    log.info(this.getServiceName() + " service end");
}

這裏的waitForRunning和Broker的刷盤以及主從複製類似，會進行超時阻塞（默認20s），也可以通過Broker發送的NOTIFY_CONSUMER_IDS_CHANGED請求將其喚醒，之後會調用doRebalance方法

RebalanceImpl的doRebalance方法：

public void doRebalance(final boolean isOrder) {
   Map<String, SubscriptionData> subTable = this.getSubscriptionInner();
    if (subTable != null) {
        for (final Map.Entry<String, SubscriptionData> entry : subTable.entrySet()) {
            final String topic = entry.getKey();
            try {
                this.rebalanceByTopic(topic, isOrder);
            } catch (Throwable e) {
                if (!topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
                    log.warn("rebalanceByTopic Exception", e);
                }
            }
        }
    }

    this.truncateMessageQueueNotMyTopic();
}

這裏就會取得copySubscription方法中說過的訂閱Topic集合，這個集合會在②中的定時任務會通過NameServer來進行更新

通過rebalanceByTopic方法，處理訂閱的Topic：

private void rebalanceByTopic(final String topic, final boolean isOrder) {
    switch (messageModel) {
        case BROADCASTING: {
            Set<MessageQueue> mqSet = this.topicSubscribeInfoTable.get(topic);
            if (mqSet != null) {
                boolean changed = this.updateProcessQueueTableInRebalance(topic, mqSet, isOrder);
                if (changed) {
                    this.messageQueueChanged(topic, mqSet, mqSet);
                    log.info("messageQueueChanged {} {} {} {}",
                        consumerGroup,
                        topic,
                        mqSet,
                        mqSet);
                }
            } else {
                log.warn("doRebalance, {}, but the topic[{}] not exist.", consumerGroup, topic);
            }
            break;
        }
        case CLUSTERING: {
            Set<MessageQueue> mqSet = this.topicSubscribeInfoTable.get(topic);
            List<String> cidAll = this.mQClientFactory.findConsumerIdList(topic, consumerGroup);
            if (null == mqSet) {
                if (!topic.startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
                    log.warn("doRebalance, {}, but the topic[{}] not exist.", consumerGroup, topic);
                }
            }

            if (null == cidAll) {
                log.warn("doRebalance, {} {}, get consumer id list failed", consumerGroup, topic);
            }

            if (mqSet != null && cidAll != null) {
                List<MessageQueue> mqAll = new ArrayList<MessageQueue>();
                mqAll.addAll(mqSet);

                Collections.sort(mqAll);
                Collections.sort(cidAll);

                AllocateMessageQueueStrategy strategy = this.allocateMessageQueueStrategy;

                List<MessageQueue> allocateResult = null;
                try {
                    allocateResult = strategy.allocate(
                        this.consumerGroup,
                        this.mQClientFactory.getClientId(),
                        mqAll,
                        cidAll);
                } catch (Throwable e) {
                    log.error("AllocateMessageQueueStrategy.allocate Exception. allocateMessageQueueStrategyName={}", strategy.getName(),
                        e);
                    return;
                }

                Set<MessageQueue> allocateResultSet = new HashSet<MessageQueue>();
                if (allocateResult != null) {
                    allocateResultSet.addAll(allocateResult);
                }

                boolean changed = this.updateProcessQueueTableInRebalance(topic, allocateResultSet, isOrder);
                if (changed) {
                    log.info(
                        "rebalanced result changed. allocateMessageQueueStrategyName={}, group={}, topic={}, clientId={}, mqAllSize={}, cidAllSize={}, rebalanceResultSize={}, rebalanceResultSet={}",
                        strategy.getName(), consumerGroup, topic, this.mQClientFactory.getClientId(), mqSet.size(), cidAll.size(),
                        allocateResultSet.size(), allocateResultSet);
                    this.messageQueueChanged(topic, mqSet, allocateResultSet);
                }
            }
            break;
        }
        default:
            break;
    }
}

這裏會根據廣播模式和集羣模式做不同的處理

廣播模式：
先根據Topic取得對應的所有消息隊列的集合

然後先通過updateProcessQueueTableInRebalance方法處理：

private boolean updateProcessQueueTableInRebalance(final String topic, final Set<MessageQueue> mqSet,
    final boolean isOrder) {
    boolean changed = false;

    Iterator<Entry<MessageQueue, ProcessQueue>> it = this.processQueueTable.entrySet().iterator();
    while (it.hasNext()) {
        Entry<MessageQueue, ProcessQueue> next = it.next();
        MessageQueue mq = next.getKey();
        ProcessQueue pq = next.getValue();

        if (mq.getTopic().equals(topic)) {
            if (!mqSet.contains(mq)) {
                pq.setDropped(true);
                if (this.removeUnnecessaryMessageQueue(mq, pq)) {
                    it.remove();
                    changed = true;
                    log.info("doRebalance, {}, remove unnecessary mq, {}", consumerGroup, mq);
                }
            } else if (pq.isPullExpired()) {
                switch (this.consumeType()) {
                    case CONSUME_ACTIVELY:
                        break;
                    case CONSUME_PASSIVELY:
                        pq.setDropped(true);
                        if (this.removeUnnecessaryMessageQueue(mq, pq)) {
                            it.remove();
                            changed = true;
                            log.error("[BUG]doRebalance, {}, remove unnecessary mq, {}, because pull is pause, so try to fixed it",
                                consumerGroup, mq);
                        }
                        break;
                    default:
                        break;
                }
            }
        }
    }

    List<PullRequest> pullRequestList = new ArrayList<PullRequest>();
    for (MessageQueue mq : mqSet) {
        if (!this.processQueueTable.containsKey(mq)) {
            if (isOrder && !this.lock(mq)) {
                log.warn("doRebalance, {}, add a new mq failed, {}, because lock failed", consumerGroup, mq);
                continue;
            }

            this.removeDirtyOffset(mq);
            ProcessQueue pq = new ProcessQueue();
            long nextOffset = this.computePullFromWhere(mq);
            if (nextOffset >= 0) {
                ProcessQueue pre = this.processQueueTable.putIfAbsent(mq, pq);
                if (pre != null) {
                    log.info("doRebalance, {}, mq already exists, {}", consumerGroup, mq);
                } else {
                    log.info("doRebalance, {}, add a new mq, {}", consumerGroup, mq);
                    PullRequest pullRequest = new PullRequest();
                    pullRequest.setConsumerGroup(consumerGroup);
                    pullRequest.setNextOffset(nextOffset);
                    pullRequest.setMessageQueue(mq);
                    pullRequest.setProcessQueue(pq);
                    pullRequestList.add(pullRequest);
                    changed = true;
                }
            } else {
                log.warn("doRebalance, {}, add new mq failed, {}", consumerGroup, mq);
            }
        }
    }

    this.dispatchPullRequest(pullRequestList);

    return changed;
}

若是消息隊列發生了更新，這裏首先在while循環中會將處理隊列中的無用的記錄刪除
而在for循環中則是爲了添加新的處理記錄，向processQueueTable添加了處理記錄，computePullFromWhere方法在PullConsumer中默認返回0，作爲nextOffset，會將該nextOffset作爲下次拉取消息的位置保存在ProcessQueue中，進而保存在processQueueTable中，作爲處理任務的記錄

之後的dispatchPullRequest方法是對於PushConsumer而言的，這裏沒有作用

回到rebalanceByTopic方法，若是發生了更新，會調用messageQueueChanged方法：

public void messageQueueChanged(String topic, Set<MessageQueue> mqAll, Set<MessageQueue> mqDivided) {
    MessageQueueListener messageQueueListener = this.defaultMQPullConsumerImpl.getDefaultMQPullConsumer().getMessageQueueListener();
    if (messageQueueListener != null) {
        try {
            messageQueueListener.messageQueueChanged(topic, mqAll, mqDivided);
        } catch (Throwable e) {
            log.error("messageQueueChanged exception", e);
        }
    }
}

這裏實際上就交給MessageQueueListener執行messageQueueChanged回調方法

集羣模式：
首先還是根據Topic得到消息隊列的集合
由於是集合模式，每個消費者會取得不同的消息，所以這裏通過findConsumerIdList方法，得到消費者的ID列表

public List<String> findConsumerIdList(final String topic, final String group) {
   String brokerAddr = this.findBrokerAddrByTopic(topic);
    if (null == brokerAddr) {
        this.updateTopicRouteInfoFromNameServer(topic);
        brokerAddr = this.findBrokerAddrByTopic(topic);
    }

    if (null != brokerAddr) {
        try {
            return this.mQClientAPIImpl.getConsumerIdListByGroup(brokerAddr, group, 3000);
        } catch (Exception e) {
            log.warn("getConsumerIdListByGroup exception, " + brokerAddr + " " + group, e);
        }
    }

    return null;
}

findBrokerAddrByTopic方法，會根據Topic選取所在集羣的一個Broker的地址（由②定時任務通過NameServer更新），若是master存在選擇master，否則隨機選擇一個slave

若是沒找到，則重新向NameServer請求更新，再找一次

當得到Broker的地址信息後，通過getConsumerIdListByGroup方法，向Broker發送請求：

public List<String> getConsumerIdListByGroup(
    final String addr,
    final String consumerGroup,
    final long timeoutMillis) throws RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException,
    MQBrokerException, InterruptedException {
    GetConsumerListByGroupRequestHeader requestHeader = new GetConsumerListByGroupRequestHeader();
    requestHeader.setConsumerGroup(consumerGroup);
    RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.GET_CONSUMER_LIST_BY_GROUP, requestHeader);

    RemotingCommand response = this.remotingClient.invokeSync(MixAll.brokerVIPChannel(this.clientConfig.isVipChannelEnabled(), addr),
        request, timeoutMillis);
    assert response != null;
    switch (response.getCode()) {
        case ResponseCode.SUCCESS: {
            if (response.getBody() != null) {
                GetConsumerListByGroupResponseBody body =
                    GetConsumerListByGroupResponseBody.decode(response.getBody(), GetConsumerListByGroupResponseBody.class);
                return body.getConsumerIdList();
            }
        }
        default:
            break;
    }

    throw new MQBrokerException(response.getCode(), response.getRemark());
}

這裏實際上就是向Broker發送了一個GET_CONSUMER_LIST_BY_GROUP請求，進行同步發送，再收到響應後，將響應中的數據，也就是消費者ID的封裝成的List返回

回到rebalanceByTopic方法，得到消費者的ID列表後
會根據分配策略進行分配，這裏默認使用的是AllocateMessageQueueAveragely
然後調用它的allocate方法，進行分配

public List<MessageQueue> allocate(String consumerGroup, String currentCID, List<MessageQueue> mqAll,
    List<String> cidAll) {
    if (currentCID == null || currentCID.length() < 1) {
        throw new IllegalArgumentException("currentCID is empty");
    }
    if (mqAll == null || mqAll.isEmpty()) {
        throw new IllegalArgumentException("mqAll is null or mqAll empty");
    }
    if (cidAll == null || cidAll.isEmpty()) {
        throw new IllegalArgumentException("cidAll is null or cidAll empty");
    }

    List<MessageQueue> result = new ArrayList<MessageQueue>();
    if (!cidAll.contains(currentCID)) {
        log.info("[BUG] ConsumerGroup: {} The consumerId: {} not in cidAll: {}",
            consumerGroup,
            currentCID,
            cidAll);
        return result;
    }

    int index = cidAll.indexOf(currentCID);
    int mod = mqAll.size() % cidAll.size();
    int averageSize =
        mqAll.size() <= cidAll.size() ? 1 : (mod > 0 && index < mod ? mqAll.size() / cidAll.size()
            + 1 : mqAll.size() / cidAll.size());
    int startIndex = (mod > 0 && index < mod) ? index * averageSize : index * averageSize + mod;
    int range = Math.min(averageSize, mqAll.size() - startIndex);
    for (int i = 0; i < range; i++) {
        result.add(mqAll.get((startIndex + i) % mqAll.size()));
    }
    return result;
}

（關於這個ID在Producer的啓動中介紹過，是在MQClientManager的getAndCreateMQClientInstance方法中，對於客戶端來說是唯一的）

由於是集羣模式，那麼這裏的Consumer也理所應當作爲其中一員，所以會檢查currentCID是否包含在集合中

接着會根據消費者的數量以及消息的數量，進行消息的分配，以此達到消費者端的負載均衡
這裏採用的是平均分配的方式，利用消息的數量以及消費者的數量就，計算出當前消費者需要消費哪部分消息

處理之外，RocketMQ中還提供其他幾種分配方式，根據需要，酌情使用

回到rebalanceByTopic方法中，在完成消息的分配後
會調用updateProcessQueueTableInRebalance方法，完成對消息隊列和處理隊列的更新，若是發生了更新，再通過messageQueueChanged方法，調用回調接口的方法，完成對消息隊列變化的通知

至此，PullConsumer的啓動結束