在《RocketMQ源碼分析之RebalanceService》中回答了消費者在第一次啓動後是如何來獲取消息這個問題，那麼在構建PullRequest(消息拉取任務)後，消費者與broker之間是如何交互來完成消息拉取任務？本篇文章就來分析消息拉取流程。在consumer端與消息拉取流程相關的服務主要是RebalanceService和PullMessageService，RebalanceService主要負責consumer端消息隊列負載均衡及構建PullRequest，PullMessageService主要負責consumer端消息拉取。下面從PullMessageService入手來分析。
PullMessageService是在consumer啓動過程中啓動MQClientInstance實例時啓動的，具體如下：
publicvoid start() throws MQClientException { synchronized (this) { switch (this.serviceState) { case CREATE_JUST:this.serviceState = ServiceState.START_FAILED;// If not specified,looking address from name serverif(null==this.clientConfig.getNamesrvAddr()) {this.mQClientAPIImpl.fetchNameServerAddr(); }// Start request-response channelthis.mQClientAPIImpl.start();// Start various schedule tasksthis.startScheduledTask();// Start pull servicethis.pullMessageService.start();// Start rebalance servicethis.rebalanceService.start();// Start push servicethis.defaultMQProducer.getDefaultMQProducerImpl().start(false); log.info("the client factory [{}] start OK",this.clientId);this.serviceState = ServiceState.RUNNING;break; case START_FAILED:thrownew MQClientException("The Factory object["+this.getClientId() +"] has been created before, and failed.",null);default:break; } }}
PullMessageService繼承ServiceThread，其本質是一個線程，在執行this.pullMessageService.start()時會執行其run方法，run方法的實現邏輯是：從pullRequestQueue中獲取一個PullRequest，如果pullRequestQueue爲空，則線程將會阻塞，直到有任務被放入，然後調用pullMessage方法進行消息拉取。
@Overridepublicvoid run() { log.info(this.getServiceName() +" service started");while(!this.isStopped()) {try{ PullRequest pullRequest =this.pullRequestQueue.take();this.pullMessage(pullRequest); }catch(InterruptedException ignored) { }catch(Exception e) { log.error("Pull Message Service Run Method exception", e); } } log.info(this.getServiceName() +" service end");}
接着再來看pullMessage(final PullRequest pullRequest)方法：在這個方法中會根據consumerGroup來獲取消費者的內部實現MQConsumerInner，然後將其強制轉換爲DefaultMQPushConsumerImpl，最後會調用DefaultMQPushConsumerImpl的pullMessage方法。在這裏面我們也不難發現PullMessageService只爲PUSH模式服務。
privatevoidpullMessage(finalPullRequest pullRequest){finalMQConsumerInner consumer =this.mQClientFactory.selectConsumer(pullRequest.getConsumerGroup());if(consumer != null) { DefaultMQPushConsumerImpl impl = (DefaultMQPushConsumerImpl) consumer; impl.pullMessage(pullRequest); }else{log.warn("No matched consumer for the PullRequest {}, drop it", pullRequest); } }
接着來看DefaultMQPushConsumerImpl的pullMessage方法：
publicvoid pullMessage(finalPullRequest pullRequest) {//從pullRequest中獲取其ProcessQueue，如果ProcessQueue沒有被丟棄則將其lastPullTimestamp屬性更新爲當前時間finalProcessQueue processQueue = pullRequest.getProcessQueue();if(processQueue.isDropped()) { log.info("the pull request[{}] is dropped.", pullRequest.toString());return; } pullRequest.getProcessQueue().setLastPullTimestamp(System.currentTimeMillis());try{/*
判斷消費者的狀態是否正常，如果消費狀態異常則將拉取任務pullRequest延遲3s再次放入到PullMessageService的拉取任務隊列中
結束本次消息拉取
*/this.makeSureStateOK(); }catch(MQClientException e) { log.warn("pullMessage exception, consumer state not ok", e);this.executePullRequestLater(pullRequest, pullTimeDelayMillsWhenException);return; }//如果消費者被掛起則將拉取任務pullRequest延遲1s再次放入到PullMessageService的拉取任務隊列中if(this.isPause()) { log.warn("consumer was paused, execute pull request later. instanceName={}, group={}",this.defaultMQPushConsumer.getInstanceName(),this.defaultMQPushConsumer.getConsumerGroup());this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_SUSPEND);return; }//消息拉取流控long cachedMessageCount = processQueue.getMsgCount().get(); long cachedMessageSizeInMiB = processQueue.getMsgSize().get() / (1024*1024);if(cachedMessageCount >this.defaultMQPushConsumer.getPullThresholdForQueue()) {this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_FLOW_CONTROL);if((queueFlowControlTimes++ %1000) ==0) { log.warn("the cached message count exceeds the threshold {}, so do flow control, minOffset={}, maxOffset={}, count={}, size={} MiB, pullRequest={}, flowControlTimes={}",this.defaultMQPushConsumer.getPullThresholdForQueue(), processQueue.getMsgTreeMap().firstKey(), processQueue.getMsgTreeMap().lastKey(), cachedMessageCount, cachedMessageSizeInMiB, pullRequest, queueFlowControlTimes); }return; }if(cachedMessageSizeInMiB >this.defaultMQPushConsumer.getPullThresholdSizeForQueue()) {this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_FLOW_CONTROL);if((queueFlowControlTimes++ %1000) ==0) { log.warn("the cached message size exceeds the threshold {} MiB, so do flow control, minOffset={}, maxOffset={}, count={}, size={} MiB, pullRequest={}, flowControlTimes={}",this.defaultMQPushConsumer.getPullThresholdSizeForQueue(), processQueue.getMsgTreeMap().firstKey(), processQueue.getMsgTreeMap().lastKey(), cachedMessageCount, cachedMessageSizeInMiB, pullRequest, queueFlowControlTimes); }return; }if(!this.consumeOrderly) {if(processQueue.getMaxSpan() >this.defaultMQPushConsumer.getConsumeConcurrentlyMaxSpan()) {this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_FLOW_CONTROL);if((queueMaxSpanFlowControlTimes++ %1000) ==0) { log.warn("the queue's messages, span too long, so do flow control, minOffset={}, maxOffset={}, maxSpan={}, pullRequest={}, flowControlTimes={}",processQueue.getMsgTreeMap().firstKey(), processQueue.getMsgTreeMap().lastKey(), processQueue.getMaxSpan(),pullRequest, queueMaxSpanFlowControlTimes); }return; } }else{if(processQueue.isLocked()) {if(!pullRequest.isLockedFirst()) {finallong offset =this.rebalanceImpl.computePullFromWhere(pullRequest.getMessageQueue()); boolean brokerBusy = offset < pullRequest.getNextOffset(); log.info("the first time to pull message, so fix offset from broker. pullRequest: {} NewOffset: {} brokerBusy: {}",pullRequest, offset, brokerBusy);if(brokerBusy) { log.info("[NOTIFYME]the first time to pull message, but pull request offset larger than broker consume offset. pullRequest: {} NewOffset: {}", pullRequest, offset); } pullRequest.setLockedFirst(true); pullRequest.setNextOffset(offset); } }else{this.executePullRequestLater(pullRequest, pullTimeDelayMillsWhenException); log.info("pull message later because not locked in broker, {}", pullRequest);return; } }/*
根據pullRequest中的topic信息，從topic的訂閱信息中獲取其對應的訂閱信息，
如果訂閱信息爲空則將拉取任務pullRequest延遲3s再次放入到PullMessageService的拉取任務隊列中並結束本次消息拉取
*/finalSubscriptionData subscriptionData =this.rebalanceImpl.getSubscriptionInner().get(pullRequest.getMessageQueue().getTopic());if(null== subscriptionData) {this.executePullRequestLater(pullRequest, pullTimeDelayMillsWhenException); log.warn("find the consumer's subscription failed, {}", pullRequest);return; }finallong beginTimestamp = System.currentTimeMillis();//構建回調pullCallback，當broker端返回response給consumer端時會執行這個回調PullCallback pullCallback = new PullCallback() {@Overridepublicvoid onSuccess(PullResult pullResult) {if(pullResult !=null) { pullResult = DefaultMQPushConsumerImpl.this.pullAPIWrapper.processPullResult(pullRequest.getMessageQueue(), pullResult, subscriptionData); switch (pullResult.getPullStatus()) { case FOUND: long prevRequestOffset = pullRequest.getNextOffset(); pullRequest.setNextOffset(pullResult.getNextBeginOffset()); long pullRT = System.currentTimeMillis() - beginTimestamp; DefaultMQPushConsumerImpl.this.getConsumerStatsManager().incPullRT(pullRequest.getConsumerGroup(), pullRequest.getMessageQueue().getTopic(), pullRT); long firstMsgOffset =Long.MAX_VALUE;if(pullResult.getMsgFoundList() ==null|| pullResult.getMsgFoundList().isEmpty()) { DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest); }else{ firstMsgOffset = pullResult.getMsgFoundList().get(0).getQueueOffset(); DefaultMQPushConsumerImpl.this.getConsumerStatsManager().incPullTPS(pullRequest.getConsumerGroup(), pullRequest.getMessageQueue().getTopic(), pullResult.getMsgFoundList().size()); boolean dispatchToConsume = processQueue.putMessage(pullResult.getMsgFoundList()); DefaultMQPushConsumerImpl.this.consumeMessageService.submitConsumeRequest( pullResult.getMsgFoundList(), processQueue, pullRequest.getMessageQueue(), dispatchToConsume);if(DefaultMQPushConsumerImpl.this.defaultMQPushConsumer.getPullInterval() >0) { DefaultMQPushConsumerImpl.this.executePullRequestLater(pullRequest, DefaultMQPushConsumerImpl.this.defaultMQPushConsumer.getPullInterval()); }else{ DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest); } }if(pullResult.getNextBeginOffset() < prevRequestOffset || firstMsgOffset < prevRequestOffset) { log.warn("[BUG] pull message result maybe data wrong, nextBeginOffset: {} firstMsgOffset: {} prevRequestOffset: {}", pullResult.getNextBeginOffset(), firstMsgOffset, prevRequestOffset); }break; case NO_NEW_MSG: pullRequest.setNextOffset(pullResult.getNextBeginOffset()); DefaultMQPushConsumerImpl.this.correctTagsOffset(pullRequest); DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest);break; case NO_MATCHED_MSG: pullRequest.setNextOffset(pullResult.getNextBeginOffset()); DefaultMQPushConsumerImpl.this.correctTagsOffset(pullRequest); DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest);break; case OFFSET_ILLEGAL: log.warn("the pull request offset illegal, {} {}", pullRequest.toString(), pullResult.toString()); pullRequest.setNextOffset(pullResult.getNextBeginOffset()); pullRequest.getProcessQueue().setDropped(true); DefaultMQPushConsumerImpl.this.executeTaskLater(new Runnable() {@Overridepublicvoid run() {try{ DefaultMQPushConsumerImpl.this.offsetStore.updateOffset(pullRequest.getMessageQueue(), pullRequest.getNextOffset(),false); DefaultMQPushConsumerImpl.this.offsetStore.persist(pullRequest.getMessageQueue()); DefaultMQPushConsumerImpl.this.rebalanceImpl.removeProcessQueue(pullRequest.getMessageQueue()); log.warn("fix the pull request offset, {}", pullRequest); }catch(Throwable e) { log.error("executeTaskLater Exception", e); } } },10000);break;default:break; } } }@Overridepublicvoid onException(Throwable e) {if(!pullRequest.getMessageQueue().getTopic().startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) { log.warn("execute the pull request exception", e); } DefaultMQPushConsumerImpl.this.executePullRequestLater(pullRequest, pullTimeDelayMillsWhenException); } }; boolean commitOffsetEnable =false; long commitOffsetValue =0L;if(MessageModel.CLUSTERING ==this.defaultMQPushConsumer.getMessageModel()) {//從內存中獲取pullRequest中MessageQueue的消費進度commitOffsetValue =this.offsetStore.readOffset(pullRequest.getMessageQueue(), ReadOffsetType.READ_FROM_MEMORY);if(commitOffsetValue >0) { commitOffsetEnable =true; } } String subExpression =null; boolean classFilter =false; SubscriptionData sd =this.rebalanceImpl.getSubscriptionInner().get(pullRequest.getMessageQueue().getTopic());if(sd !=null) {if(this.defaultMQPushConsumer.isPostSubscriptionWhenPull() && !sd.isClassFilterMode()) { subExpression = sd.getSubString(); } classFilter = sd.isClassFilterMode(); }//構建消息拉取的系統標記int sysFlag = PullSysFlag.buildSysFlag( commitOffsetEnable,// commitOffsettrue,// suspendsubExpression !=null,// subscriptionclassFilter// class filter);try{//與broker端交互this.pullAPIWrapper.pullKernelImpl( pullRequest.getMessageQueue(), subExpression, subscriptionData.getExpressionType(), subscriptionData.getSubVersion(), pullRequest.getNextOffset(),this.defaultMQPushConsumer.getPullBatchSize(), sysFlag, commitOffsetValue, BROKER_SUSPEND_MAX_TIME_MILLIS, CONSUMER_TIMEOUT_MILLIS_WHEN_SUSPEND, CommunicationMode.ASYNC, pullCallback ); }catch(Exception e) { log.error("pullKernelImpl exception", e);this.executePullRequestLater(pullRequest, pullTimeDelayMillsWhenException); }}
pullKernelImpl方法具體如下：
publicPullResultpullKernelImpl(finalMessageQueue mq,finalString subExpression,finalString expressionType,finallongsubVersion,finallongoffset,finalintmaxNums,finalintsysFlag,finallongcommitOffset,finallongbrokerSuspendMaxTimeMillis,finallongtimeoutMillis,finalCommunicationMode communicationMode,finalPullCallback pullCallback )throwsMQClientException, RemotingException, MQBrokerException, InterruptedException{/*
根據brokerName、brokerId從mQClientFactory中獲取broker的地址
在RocketMQ中相同名稱的broker會有多個（主broker和從broker），但是brokerId會不一樣
在每次拉取消息後會給出下次拉取消息時的建議，即從主broker上拉取還是從從broker上拉取
*/FindBrokerResult findBrokerResult =this.mQClientFactory.findBrokerAddressInSubscribe(mq.getBrokerName(),this.recalculatePullFromWhichNode(mq),false);//如果findBrokerResult爲空，則首先會更新客戶端topic路由信息表//然後再次執行findBrokerAddressInSubscribe方法獲取broker的地址if(null== findBrokerResult) {this.mQClientFactory.updateTopicRouteInfoFromNameServer(mq.getTopic()); findBrokerResult =this.mQClientFactory.findBrokerAddressInSubscribe(mq.getBrokerName(),this.recalculatePullFromWhichNode(mq),false); }if(findBrokerResult !=null) { {// check versionif(!ExpressionType.isTagType(expressionType) && findBrokerResult.getBrokerVersion() < MQVersion.Version.V4_1_0_SNAPSHOT.ordinal()) {thrownewMQClientException("The broker["+ mq.getBrokerName() +", "+ findBrokerResult.getBrokerVersion() +"] does not upgrade to support for filter message by "+ expressionType,null); } }intsysFlagInner = sysFlag;if(findBrokerResult.isSlave()) { sysFlagInner = PullSysFlag.clearCommitOffsetFlag(sysFlagInner); }//構建PullMessageRequestHeaderPullMessageRequestHeader requestHeader =newPullMessageRequestHeader(); requestHeader.setConsumerGroup(this.consumerGroup); requestHeader.setTopic(mq.getTopic()); requestHeader.setQueueId(mq.getQueueId()); requestHeader.setQueueOffset(offset); requestHeader.setMaxMsgNums(maxNums); requestHeader.setSysFlag(sysFlagInner); requestHeader.setCommitOffset(commitOffset); requestHeader.setSuspendTimeoutMillis(brokerSuspendMaxTimeMillis); requestHeader.setSubscription(subExpression); requestHeader.setSubVersion(subVersion); requestHeader.setExpressionType(expressionType);/*
如果消息過濾的模式是類過濾，則根據topic、broker地址找到註冊在broker上的FilterServer地址，從FilterServer上拉取信息,
否則從broker上拉取信息
*/String brokerAddr = findBrokerResult.getBrokerAddr();if(PullSysFlag.hasClassFilterFlag(sysFlagInner)) { brokerAddr = computPullFromWhichFilterServer(mq.getTopic(), brokerAddr); } PullResult pullResult =this.mQClientFactory.getMQClientAPIImpl().pullMessage( brokerAddr, requestHeader, timeoutMillis, communicationMode, pullCallback);returnpullResult; }thrownewMQClientException("The broker["+ mq.getBrokerName() +"] not exist",null); }
進入public PullResult pullMessage(final String addr, final PullMessageRequestHeader requestHeader, final long timeoutMillis,final CommunicationMode communicationMode,final PullCallback pullCallback) 方法中會發現客戶端向broker發送的請求類型是“RequestCode.PULL_MESSAGE”，通過在代碼中查找可以發現broker端處理該類型請求的是PullMessageProcessor的processRequest方法。
/**
* PullMessageProcessor
*/this.remotingServer.registerProcessor(RequestCode.PULL_MESSAGE,this.pullMessageProcessor,this.pullMessageExecutor);this.pullMessageProcessor.registerConsumeMessageHook(consumeMessageHookList);
下面來看broker端是如何處理客戶端發送的拉取消息的請求。
1.構建返回給consumer端的response並解析發送到broker端的request
2.檢查broker的權限是否可讀，如果不可讀則將response的code設置爲ResponseCode.SUBSCRIPTION_GROUP_NOT_EXIST並返回給consumer端
3.在broker端獲取consumer消費組的信息，如果配置信息中consumeEnable屬性爲false，則將response的code設置爲ResponseCode.NO_PERMISSION並返回給consumer端
4.從請求中獲取消息拉取時設置的系統標記
5.在broker端獲取消息topic的配置信息，如果配置信息爲空則將response的code設置爲ResponseCode.TOPIC_NOT_EXIST並返回給consumer端
6.檢查topic的權限是否可讀，如果不可讀則將response的code設置爲ResponseCode.NO_PERMISSION並返回給consumer端
7.檢查待拉取信息的MessageQueue的queueid是否合法，如果不合法則將response的code設置爲ResponseCode.SYSTEM_ERROR並返回給consumer端
8.根據topic、消息過濾表達式構建訂閱消息實體，如果不是TAG模式則構建過濾數據consumerFilterData
9.構建消息過濾對象messageFilter
10.根據requestHeader中消費者的消費組名稱、topic名稱、MessageQueue的queueId、待拉取信息的ConsumeQueue的邏輯偏移量、最大拉取消息條數和消息過濾器來查找消息。getMessage方法中會計算出下次拉取任務的開始偏移量nextBeginOffset
11.如果獲取到的getMessageResult不爲空，則在response中設置nextBeginOffset、minOffset、maxOffset
12.如果從節點中包含下次拉取的偏移量則設置爲下一次拉取任務的brokerId
13.根據getMessageResult的status來設置response中的code，其對應關係如下：
getMessageResult statusResponseCodeFOUNDSUCCESSMESSAGE_WAS_REMOVINGPULL_RETRY_IMMEDIATELYNO_MATCHED_LOGIC_QUEUE、NO_MESSAGE_IN_QUEUE、OFFSET_OVERFLOW_BADLY 、OFFSET_TOO_SMALLPULL_OFFSET_MOVEDNO_MATCHED_MESSAGEPULL_RETRY_IMMEDIATELYOFFSET_FOUND_NULL 、OFFSET_OVERFLOW_ONEPULL_NOT_FOUND
14.如果當前節點是主節點並且commitlog標記可用，則會觸發更新消息消費進度
15.將response返回給consumer端
broker將response返回給consumer端時會回調PullCallBack的onSuccess或者onException，PullCallBack就是pullMessage(final PullRequest pullRequest) 方法中創建的。回調PullCallBack的方法如下：
this.remotingClient.invokeAsync(addr, request, timeoutMillis,newInvokeCallback() {@OverridepublicvoidoperationComplete(ResponseFuture responseFuture){ RemotingCommand response = responseFuture.getResponseCommand();if(response !=null) {try{ PullResult pullResult = MQClientAPIImpl.this.processPullResponse(response);assertpullResult !=null; pullCallback.onSuccess(pullResult); }catch(Exception e) { pullCallback.onException(e); } }else{if(!responseFuture.isSendRequestOK()) { pullCallback.onException(newMQClientException("send request failed to "+ addr +". Request: "+ request, responseFuture.getCause())); }elseif(responseFuture.isTimeout()) { pullCallback.onException(newMQClientException("wait response from "+ addr +" timeout :"+ responseFuture.getTimeoutMillis() +"ms"+". Request: "+ request, responseFuture.getCause())); }else{ pullCallback.onException(newMQClientException("unknown reason. addr: "+ addr +", timeoutMillis: "+ timeoutMillis +". Request: "+ request, responseFuture.getCause())); } } } });
接下來看看consumer端收到broker返回的response會如何處理？
1.根據broker端返回的response將其處理成PullResult，這一過程調用的是processPullResponse方法，該方法會進行狀態碼轉換、構建PullResult對象。
response codepull statusSUCCESSFOUNDPULL_NOT_FOUNDNO_NEW_MSGPULL_RETRY_IMMEDIATELYNO_MATCHED_MSGPULL_OFFSET_MOVEDOFFSET_ILLEGAL
2.根據pullResult更新下一次拉取的偏移量，如果pullResult中的msgFoundList爲空則立刻把PullRequest放入PullMessageService的pullRequestQueue隊列中
3.將拉取到的消息放入processQueue中，然後再將消息提交到ConsumeMessageQueue（ConsumeMessageQueue分爲兩種，分別是ConsumeMessageConcurrentlyService和ConsumeMessageOrderlyService）中用於consumer消費
4.如果pullInterval大於0，則將pullRequest延遲pullInterval毫秒後放入PullMessageService的pullRequestQueue隊列中，這樣形成持續拉取消息流程
最後，總結下消息拉取流程，該流程總體上分爲三步：
1.consumer端封裝消息拉取請求PullRequest並將其發送給broker
2.broker根據請求查找並返回消息給consumer端
3.consumer端將返回的消息消費
那麼consumer端獲取到消息後如何進行消費，下篇文章再分析。