RabbitMQ（三）：工作隊列

一、工作隊列

官方文檔：http://www.rabbitmq.com/tutorials/tutorial-two-java.html

簡單隊列不足：不支持多個消費者

即一個生產者可以對應多個消費者同時消費，相比簡單隊列支持多消費者。因爲實際工作中，生產者服務一般都是很簡單的業務邏輯處理之後就發送到隊列，消費者接收到隊列的消息之後，進行復雜的業務邏輯處理，所以一般都是多個消費者進行處理。如果是一個消費者進行處理，那麼隊列會積壓很多消息。

工作隊列分兩種情況：

輪詢分發

不管消費者處理速度性能快慢，每個消費者都是按順序分別每個拿一個的原則，比如3個消費者, 消費者1拿1個，然後消費者2拿一個，然後消費者3拿一個，然後消費者1開始拿，即使中間有消費者已經處理完了，也必須等待其他消費者都拿完一個才能消費到。
公平分發

根據消費者處理性能，性能好的消費的數據量多，性能差的消費的數據量少。

如上所示，如果配置有用戶名密碼以及vhost，則配置即可。

二、輪詢分發

一個生產者，兩個消費者，其中消費者處理只要1s，消費者2 處理需要2s

連接RabbitMQ工具類

package cn.saytime.rabbitmq.util;

import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * RabbitMQ連接工具類
 */
public class ConnectionUtil {

    private static final String host = "192.168.239.128";
    private static final int port = 5672;

    /**
     * 獲取RabbitMQ Connection連接
     * @return
     * @throws IOException
     * @throws TimeoutException
     */
    public static Connection getConnection() throws IOException, TimeoutException {
        ConnectionFactory connectionFactory = new ConnectionFactory();
        connectionFactory.setHost(host);
        connectionFactory.setPort(port);

//        connectionFactory.setUsername("test");
//        connectionFactory.setPassword("123456");
//        connectionFactory.setVirtualHost("/vhost_test");

        return connectionFactory.newConnection();
    }
}

生產者

package cn.saytime.rabbitmq.work;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 生產者
 */
public class Send {

    private static final String QUEUE_NAME = "test_work_queue";

    public static void main(String[] args) throws IOException, TimeoutException {
        // 獲取連接
        Connection connection = ConnectionUtil.getConnection();
        // 從連接開一個通道
        Channel channel = connection.createChannel();
        // 申明這個通道連接的隊列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        for (int i = 0; i < 20; i++) {
            String message = "Hello RabbitMQ " + i;
            // 發送消息
            channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
            System.out.println(" [x] Sent '" + message + "'");
        }

        // 關閉通道和連接
        channel.close();
        connection.close();
    }

}

消費者1

package cn.saytime.rabbitmq.work;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 消費者
 */
public class Recv {

    private static final String QUEUE_NAME = "test_work_queue";

    public static void main(String[] args) throws IOException, TimeoutException {

        // 獲取連接
        Connection connection = ConnectionUtil.getConnection();

        // 打開通道
        Channel channel = connection.createChannel();

        // 申明要消費的隊列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        // 創建一個回調的消費者處理類
        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
                // 接收到的消息
                String message = new String(body);
                System.out.println(" [1] Received '" + message + "'");

                try {
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                } finally {
                    System.out.println(" [1] done ");
                }
            }
        };

        // 消費消息
        channel.basicConsume(QUEUE_NAME, true, consumer);

    }

}

消費者2

package cn.saytime.rabbitmq.work;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 消費者
 */
public class Recv2 {

    private static final String QUEUE_NAME = "test_work_queue";

    public static void main(String[] args) throws IOException, TimeoutException {

        // 獲取連接
        Connection connection = ConnectionUtil.getConnection();

        // 打開通道
        Channel channel = connection.createChannel();

        // 申明要消費的隊列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        // 創建一個回調的消費者處理類
        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
                // 接收到的消息
                String message = new String(body);
                System.out.println(" [2] Received '" + message + "'");

                try {
                    Thread.sleep(2000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                } finally {
                    System.out.println(" [2] done ");
                }
            }
        };

        // 消費消息
        channel.basicConsume(QUEUE_NAME, true, consumer);

    }

}

測試

先啓動兩個消費者Recv、Recv2
再啓動生產者Send

生產者控制檯：

 [x] Sent 'Hello RabbitMQ 0'
 [x] Sent 'Hello RabbitMQ 1'
 [x] Sent 'Hello RabbitMQ 2'
 [x] Sent 'Hello RabbitMQ 3'
 [x] Sent 'Hello RabbitMQ 4'
 [x] Sent 'Hello RabbitMQ 5'
 [x] Sent 'Hello RabbitMQ 6'
 [x] Sent 'Hello RabbitMQ 7'
 [x] Sent 'Hello RabbitMQ 8'
 [x] Sent 'Hello RabbitMQ 9'
 [x] Sent 'Hello RabbitMQ 10'
 [x] Sent 'Hello RabbitMQ 11'
 [x] Sent 'Hello RabbitMQ 12'
 [x] Sent 'Hello RabbitMQ 13'
 [x] Sent 'Hello RabbitMQ 14'
 [x] Sent 'Hello RabbitMQ 15'
 [x] Sent 'Hello RabbitMQ 16'
 [x] Sent 'Hello RabbitMQ 17'
 [x] Sent 'Hello RabbitMQ 18'
 [x] Sent 'Hello RabbitMQ 19'

Process finished with exit code 0

消費者1：

 [1] Received 'Hello RabbitMQ 0'
 [1] done 
 [1] Received 'Hello RabbitMQ 2'
 [1] done 
 [1] Received 'Hello RabbitMQ 4'
 [1] done 
 [1] Received 'Hello RabbitMQ 6'
 [1] done 
 [1] Received 'Hello RabbitMQ 8'
 [1] done 
 [1] Received 'Hello RabbitMQ 10'
 [1] done 
 [1] Received 'Hello RabbitMQ 12'
 [1] done 
 [1] Received 'Hello RabbitMQ 14'
 [1] done 
 [1] Received 'Hello RabbitMQ 16'
 [1] done 
 [1] Received 'Hello RabbitMQ 18'
 [1] done

消費者2：

 [2] Received 'Hello RabbitMQ 1'
 [2] done 
 [2] Received 'Hello RabbitMQ 3'
 [2] done 
 [2] Received 'Hello RabbitMQ 5'
 [2] done 
 [2] Received 'Hello RabbitMQ 7'
 [2] done 
 [2] Received 'Hello RabbitMQ 9'
 [2] done 
 [2] Received 'Hello RabbitMQ 11'
 [2] done 
 [2] Received 'Hello RabbitMQ 13'
 [2] done 
 [2] Received 'Hello RabbitMQ 15'
 [2] done 
 [2] Received 'Hello RabbitMQ 17'
 [2] done 
 [2] Received 'Hello RabbitMQ 19'
 [2] done

可以發現消費者1 消費的數字全是整數，消費者2消費的全是奇數。

那麼如果我事先啓動三個消費者了，那麼結果如何了？

因爲20/3 除不了，如果消費者1,2,3按順序啓動，那麼消費者1, 2會消費7條數據，消費者3消費6條，其中

消費者1 消費 0, 3, 6, 9, 12, 15, 18
消費者2 消費 1, 4, 7, 10, 13, 16, 19
消費者3 消費 2, 5, 8, 11, 14, 17

注意點：

如果生產者一次性發送完消息之後，再依次啓動消費者1, 2, 3，之後只有消費者1 能消費到數據，消費者都啓動之後，再生產的消息就會輪詢分發到消費者1, 2, 3

三、公平分發

官方示例圖：

因爲生產者發送消息到隊列之後，隊列不知道消費者有沒有處理完，所以多個消費者同時訂閱同一個Queue中的消息，Queue中的消息會被平攤給多個消費者。爲了實現公平分發，我們需要告訴隊列，每次發一個給我，然後我反饋給你我有沒有處理完，處理完了你再發一條給我。

在默認輪詢分發的基礎上，要實現公平分發，需要兩點：

限制發送給同一個消費者不得超過1條消息，在這個消費者確認消息之前，不會發送下一條消息給這個消費者

int prefetchCount = 1;
channel.basicQos(prefetchCount);

默認自動應答改成手動應答


關閉自動應答
boolean autoAck = false;
channel.basicConsume(QUEUE_NAME, autoAck, consumer);

手動應答
channel.basicAck(envelope.getDeliveryTag(), false);

DeliveryTag 用來標識信道中投遞的消息， RabbitMQ 推送消息給 Consumer 時，會附帶一個 Delivery Tag，以便 Consumer 可以在消息確認時告訴 RabbitMQ 到底是哪條消息被確認了。

生產者

package cn.saytime.rabbitmq.workfair;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 生產者
 */
public class Send {

    private static final String QUEUE_NAME = "test_workfair_queue";

    public static void main(String[] args) throws IOException, TimeoutException {
        // 獲取連接
        Connection connection = ConnectionUtil.getConnection();
        // 從連接開一個通道
        Channel channel = connection.createChannel();
        // 申明這個通道連接的隊列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        channel.basicQos(1);

        for (int i = 0; i < 20; i++) {
            String message = "Hello RabbitMQ " + i;
            // 發送消息
            channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
            System.out.println(" [x] Sent '" + message + "'");
            try {
                Thread.sleep(i*100);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }

        // 關閉通道和連接
        channel.close();
        connection.close();
    }

}

消費者1

package cn.saytime.rabbitmq.workfair;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 消費者
 */
public class Recv {

    private static final String QUEUE_NAME = "test_workfair_queue";

    public static void main(String[] args) throws IOException, TimeoutException {

        // 獲取連接
        Connection connection = ConnectionUtil.getConnection();

        // 打開通道
        final Channel channel = connection.createChannel();

        // 申明要消費的隊列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        // 這樣RabbitMQ就會使得每個Consumer在同一個時間點最多處理一個Message。換句話說，在接收到該Consumer的ack前，他它不會將新的Message分發給它。
        channel.basicQos(1);

        // 創建一個回調的消費者處理類
        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
                // 接收到的消息
                String message = new String(body);
                System.out.println(" [1] Received '" + message + "'");

                try {
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                } finally {
                    System.out.println(" [1] done ");
                    channel.basicAck(envelope.getDeliveryTag(), false);
                }
            }
        };

        // 消費消息
        channel.basicConsume(QUEUE_NAME, false, consumer);

    }

}

消費者2

package cn.saytime.rabbitmq.workfair;

import cn.saytime.rabbitmq.util.ConnectionUtil;
import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

/**
 * 消費者
 */
public class Recv2 {

    private static final String QUEUE_NAME = "test_workfair_queue";

    public static void main(String[] args) throws IOException, TimeoutException {

        // 獲取連接
        Connection connection = ConnectionUtil.getConnection();

        // 打開通道
        Channel channel = connection.createChannel();

        // 申明要消費的隊列
        channel.queueDeclare(QUEUE_NAME, false, false, false, null);

        // 這樣RabbitMQ就會使得每個Consumer在同一個時間點最多處理一個Message。換句話說，在接收到該Consumer的ack前，他它不會將新的Message分發給它。
        channel.basicQos(1);

        // 創建一個回調的消費者處理類
        Consumer consumer = new DefaultConsumer(channel) {
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body) throws IOException {
                // 接收到的消息
                String message = new String(body);
                System.out.println(" [2] Received '" + message + "'");

                try {
                    Thread.sleep(2000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                } finally {
                    System.out.println(" [2] done ");
                    channel.basicAck(envelope.getDeliveryTag(), false);
                }
            }
        };

        // 消費消息
        channel.basicConsume(QUEUE_NAME, false, consumer);

    }

}

測試

消費者1:

 [1] Received 'Hello RabbitMQ 0'
 [1] done 
 [1] Received 'Hello RabbitMQ 2'
 [1] done 
 [1] Received 'Hello RabbitMQ 4'
 [1] done 
 [1] Received 'Hello RabbitMQ 5'
 [1] done 
 [1] Received 'Hello RabbitMQ 7'
 [1] done 
 [1] Received 'Hello RabbitMQ 8'
 [1] done 
 [1] Received 'Hello RabbitMQ 10'
 [1] done 
 [1] Received 'Hello RabbitMQ 11'
 [1] done 
 [1] Received 'Hello RabbitMQ 13'
 [1] done 
 [1] Received 'Hello RabbitMQ 14'
 [1] done 
 [1] Received 'Hello RabbitMQ 15'
 [1] done 
 [1] Received 'Hello RabbitMQ 17'
 [1] done 
 [1] Received 'Hello RabbitMQ 19'
 [1] done

消費者2:

 [2] Received 'Hello RabbitMQ 1'
 [2] done 
 [2] Received 'Hello RabbitMQ 3'
 [2] done 
 [2] Received 'Hello RabbitMQ 6'
 [2] done 
 [2] Received 'Hello RabbitMQ 9'
 [2] done 
 [2] Received 'Hello RabbitMQ 12'
 [2] done 
 [2] Received 'Hello RabbitMQ 16'
 [2] done 
 [2] Received 'Hello RabbitMQ 18'
 [2] done

顯然消費者1處理速度只要1s，所以消費的記錄數要比消費者2要多很多。表示確實是公平分發。

注意點：

當關閉自動應答autoAck=false之後，在消費者處理消費數據之後一定要對消息進行手動反饋處理，可以是basicAck，也可以是basicNack， basicReject


BasicReject一次只能拒絕接收一個消息，而BasicNack方法可以支持一次0個或多個消息的拒收，並且也可以設置是否requeue。


// 拒絕當前消息，並使這條消息重新返回到隊列中
channel.basicNack(envelope.getDeliveryTag(), false, true);
相當於
channel.basicReject(envelope.getDeliveryTag(), true);

RabbitMQ（三）：工作隊列

一、工作隊列

二、輪詢分發

生產者

消費者1

消費者2

測試

三、公平分發

生產者

消費者1

消費者2

測試

開源高性能結構化日誌模塊NanoLog

杭州的 IT 崩盤了麼？

【簡寫Mybatis-02】註冊機的實現以及SqlSession處理

手繪二維碼

.NET藉助虛擬網卡實現一個簡單異地組網工具

Kafka（一）：初識Kafka

Kafka（三）：Kafka集羣

RabbitMQ（一）：RabbitMQ安裝

RabbitMQ（四）：訂閱模式

RabbitMQ（三）：工作隊列

Mac下配置sublime實現LaTeX

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結