RocketMQ：事務消息

什麼是事務消息

What is transactional message?
It can be thought of as a two-phase commit message implementation to ensure eventual consistency in distributed system. Transactional message ensures that the execution of local transaction and the sending of message can be performed atomically.

就是說是一種兩階段提交來實現分佈式系統中的最終一致性。事務消息保證了本地事務和消息發送的原子性。

使用約束

(1) Messages of the transactional have no schedule and batch support.
(2) In order to avoid a single message being checked too many times and lead to half queue message accumulation， we limited the number of checks for a single message to 15 times by default, but users can change this limit by change the “transactionCheckMax” parameter in the configuration of the broker， if one message has been checked over “transactionCheckMax” times， broker will discard this message and print an error log at the same time by default. Users can change this behavior by override the “AbstractTransactionCheckListener” class.
(3) A transactional message will be checked after a certain period of time that determined by parameter “transactionTimeout” in the configuration of the broker. And users also can change this limit by set user property “CHECK_IMMUNITY_TIME_IN_SECONDS” when sending transactional message, this parameter takes precedence over the “transactionMsgTimeout” parameter.
(4) A transactional message maybe checked or consumed more than once.
(5) Committed message reput to the user’s target topic may fail. Currently, it depends on the log record. High availability is ensured by the high availability mechanism of RocketMQ itself. If you want to ensure that the transactional message isn’t lost and the transaction integrity is guaranteed, it is recommended to use synchronous double write. mechanism.
(6) Producer IDs of transactional messages cannot be shared with producer IDs of other types of messages. Unlike other types of message, transactional messages allow backward queries. MQ Server query clients by their Producer IDs.

翻譯官網如下：

1. 事務消息不支持延時發送和批量發送。
2. 爲了避免單條消息被檢查太多次數，導致隊列消息阻塞，默認限制單挑消息檢查次數上限爲15，你可以修改transactionCheckMax
參數。如果達到最大次數，broker默認會丟棄消息，並打印錯誤日誌。你可以修改這個行爲：通過重寫AbstractTransactionCheckListener
類。
3. 事務消息被檢查再一段時間之後，時間是transactionTimeout配置的，當發送事務消息時用戶可以改變該值通過CHECK_IMMUNITY_TIME_IN_SECONDS，該參數優先於transactionMsgTimeout。
4. 事務消息可以被檢查或者消費多次。
5. 被提交的消息重新放到目標topic可能會失敗。現在，取決於日誌記錄。RocketMQ的高可用機制保證了高可用，如果你想確保事務消息不會丟失，推薦使用同步雙寫機制。
6. 事務消息的id不能和其他消息類型的生產者id共享。不像其他類型的消息，事務消息可以回查。可以通過Producer IDS進行查詢。

事務架構設計

事務消息的三種狀態

(1) TransactionStatus.CommitTransaction: commit transaction，it means that allow consumers to consume this message.

TransactionStatus.CommitTransaction
提交狀態，消費者可以消費消息。

(2) TransactionStatus.RollbackTransaction: rollback transaction，it means that the message will be deleted and not allowed to consume.

TransactionStatus.RollbackTransaction
回滾狀態，消息會被刪除，消息不會被消費。

(3) TransactionStatus.Unknown: intermediate state，it means that MQ is needed to check back to determine the status.

TransactionStatus.Unknown
未知狀態，是一種中間狀態，需要檢查是否提交或者回滾。

架構設計

舉一個實際案例：

1. 生產者執行本地事務，修改訂單支付狀態，並提交事務；
2. 生產者發送事務消息到broker上，消息發送到broker上在沒有確認之前，消息對於consumer是不可見狀態；
3. 生產者確認事務消息，使得發送到broker上的消息對於消費者可見；
4. 消費者獲取到消息消費，完成後執行ack進行確認；

上面的流程可能存在一個問題，生產者本地事務成功後，發送事務確認消息到broker上失敗了，導致消息一直是unKnown狀態，消費者獲取不到消息，該怎麼辦？
Rocket提供了消息會查機制，如果消息一直處於中間狀態，broker會發起重試去查詢broker上這個事務的處理狀態，一旦發現事務處理成功，則把當前消息設置爲可見。否則就是回滾。上面使用約束第2條可以設置check最大次數。

實踐

以官網的例子：

發送事務消息

Create the transactional producer
Use TransactionMQProducer class to create producer client, and specify a unique producerGroup, and you can set up a custom thread pool to process check requests. After executing the local transaction, you need to reply to MQ according to the execution result，and the reply status is described in the above section.

使用TransactionMQProducer創建發送者對象，指定唯一的producerGroup，你也可以設置自定義的線程池來處理check請求。在執行本地事務後，需要回復mq根據事務執行結果。

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.common.message.MessageExt;
import java.util.List;

public class TransactionProducer {
    public static void main(String[] args) throws MQClientException, InterruptedException {
    
        TransactionListener transactionListener = new TransactionListenerImpl();
        //指定唯一的發送組
        TransactionMQProducer producer = new TransactionMQProducer("please_rename_unique_group_name");
        //自定義線程池處理check請求
        ExecutorService executorService = new ThreadPoolExecutor(2, 5, 100, TimeUnit.SECONDS, new                    ArrayBlockingQueue<Runnable>(2000), new ThreadFactory() {
            @Override
            public Thread newThread(Runnable r) {
                Thread thread = new Thread(r);
                thread.setName("client-transaction-msg-check-thread");
                return thread;
            }
        });

        producer.setExecutorService(executorService);
        producer.setTransactionListener(transactionListener);
        producer.start();

        String[] tags = new String[] {"TagA", "TagB", "TagC", "TagD", "TagE"};
        for (int i = 0; i < 10; i++) {
            try {
                Message msg = new Message("TopicTest1234", tags[i % tags.length], "KEY" + i,
                        ("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
                //發送事務消息        
                SendResult sendResult = producer.sendMessageInTransaction(msg, null);
                
                System.out.printf("%s%n", sendResult);

                Thread.sleep(10);
            } catch (MQClientException | UnsupportedEncodingException e) {
                e.printStackTrace();
            }
        }

        for (int i = 0; i < 100000; i++) {
            Thread.sleep(1000);
        }
        producer.shutdown();
    }
}

實現接口TransactionListener

Implement the TransactionListener interface
The “executeLocalTransaction” method is used to execute local transaction when send half message succeed. It returns one of three transaction status mentioned in the previous section.The “checkLocalTransaction” method is used to check the local transaction status and respond to MQ check requests. It also returns one of three transaction status mentioned in the previous section.

實現TransactionListener接口。
executeLocalTransaction方法用來執行本地事務，當發送消息成功一半，會返回三個事務狀態TransactionStatus之一。
checkLocalTransaction方法用來檢查本地事務狀態，響應回查請求，返回三個事務狀態TransactionStatus之一。

 public class TransactionListenerImpl implements TransactionListener {
 
       private AtomicInteger transactionIndex = new AtomicInteger(0);
   
       private ConcurrentHashMap<String, Integer> localTrans = new ConcurrentHashMap<>();
   
        //執行本地事務方法
       @Override
       public LocalTransactionState executeLocalTransaction(Message msg, Object arg) {
           int value = transactionIndex.getAndIncrement();
           int status = value % 3;
           localTrans.put(msg.getTransactionId(), status);
           //返回中間狀態，事務消息對消費者不可見
           return LocalTransactionState.UNKNOW;
       }
   
         //回查本地事務方法，提供給broker回調
       @Override
       public LocalTransactionState checkLocalTransaction(MessageExt msg) {
           Integer status = localTrans.get(msg.getTransactionId());
           if (null != status) {
               switch (status) {
                   case 0:
                       return LocalTransactionState.UNKNOW;
                   case 1:
                       return LocalTransactionState.COMMIT_MESSAGE;
                   case 2:
                       return LocalTransactionState.ROLLBACK_MESSAGE;
               }
           }
           return LocalTransactionState.COMMIT_MESSAGE;
       }
   }