RocketMQ 源碼閱讀 ---- Tag 過濾

零、簡介

RocketMQ 消息過濾分成 TAG過濾和 SQL Filter 過濾，SQL Filter是在服務端處理，會影響 MQ 的性能一般不建議使用，語法比較靈活，實現方式也相對複雜一些。Tags 過濾實現比較簡單，在客戶端實現。這樣就有一個問題，如果某個 TOPIC 消息非常多，主要消費這個 TOPIC 應用A要是 500 臺，另一個只想消費部分 Tag 消息的應用B只有 2 臺，500臺能承接的海量消息會也會到只有2臺機器的應用B，因爲機器少，可能應用B就有消息堆積的報警。 （tips：一個 24C-125G內存-60G 機器部署MQ服務端大概能支撐5W左右的 TPS）

一、源碼解析

DefaultMQPushConsumerImpl 類爲消息消費類

public void pullMessage(final PullRequest pullRequest) {
        .......

            if (this.isPause()) {

                    final SubscriptionData subscriptionData = (SubscriptionData)this.rebalanceImpl.getSubscriptionInner().get(pullRequest.getMessageQueue().getTopic());
                    if (null == subscriptionData) {
                        this.executePullRequestLater(pullRequest, 3000L);
                        this.log.warn("find the consumer's subscription failed, {}", pullRequest);
                    } else {
                        final long beginTimestamp = System.currentTimeMillis();
                        PullCallback pullCallback = new PullCallback() {
                            public void onSuccess(PullResult pullResult) {
                                if (pullResult != null) {
                                    pullResult = DefaultMQPushConsumerImpl.this.pullAPIWrapper.processPullResult(pullRequest.getMessageQueue(), pullResult, subscriptionData);
                                    switch(pullResult.getPullStatus()) {
                                    case FOUND:
                                        

      ......                                      
}

processPullResult 就是拉取處理消息的真正位置

public PullResult processPullResult(MessageQueue mq, PullResult pullResult, SubscriptionData subscriptionData) {
        PullResultExt pullResultExt = (PullResultExt)pullResult;
        this.updatePullFromWhichNode(mq, pullResultExt.getSuggestWhichBrokerId());
        if (PullStatus.FOUND == pullResult.getPullStatus()) {
            ByteBuffer byteBuffer = ByteBuffer.wrap(pullResultExt.getMessageBinary());
            List<MessageExt> msgList = MessageDecoder.decodesBatch(byteBuffer, this.mQClientFactory.getClientConfig().isDecodeReadBody(), this.mQClientFactory.getClientConfig().isDecodeDecompressBody(), true);
            List<MessageExt> msgListFilterAgain = msgList;
            Iterator i$;
            MessageExt msg;
            if (!subscriptionData.getTagsSet().isEmpty() && !subscriptionData.isClassFilterMode()) {
                msgListFilterAgain = new ArrayList(msgList.size());
                i$ = msgList.iterator();

                while(i$.hasNext()) {
                    msg = (MessageExt)i$.next();
                    if (msg.getTags() != null && subscriptionData.getTagsSet().contains(msg.getTags())) { // tags 過濾
                        ((List)msgListFilterAgain).add(msg);
                    }
                }
            }

            if (this.hasHook()) {
                FilterMessageContext filterMessageContext = new FilterMessageContext();
                filterMessageContext.setUnitMode(this.unitMode);
                filterMessageContext.setMsgList((List)msgListFilterAgain);
                this.executeHook(filterMessageContext);
            }

            i$ = ((List)msgListFilterAgain).iterator();

            while(i$.hasNext()) {
                msg = (MessageExt)i$.next();
                MessageAccessor.putProperty(msg, "MIN_OFFSET", Long.toString(pullResult.getMinOffset()));
                MessageAccessor.putProperty(msg, "MAX_OFFSET", Long.toString(pullResult.getMaxOffset()));
            }

            pullResultExt.setMsgFoundList((List)msgListFilterAgain);
        }

        pullResultExt.setMessageBinary((byte[])null);
        return pullResult;
    }

可以看到上述代碼，是判斷 MetaQ 消息體中是否 contains Tags，如果有就放進結果集。