Talking RabbitMQ with Python and Kombu

目錄

What is RabbitMQ?

Now On To The Code

使用Python3和Kombu來建立一個生產者

創建消費者

---

• 這是一個系列文章裏面的的第一篇。主要是關於RabbitMQ的介紹和如何使用Python和Kombu搭建起一個消息隊列的系列文章。
• https://medium.com/python-pandemonium/talking-to-rabbitmq-with-python-and-kombu-6cbee93b1298 （需要翻牆）
• 中文翻譯：https://www.jianshu.com/p/35a6db9b16f7

 

參考：

深入理解AMQP協議

What is RabbitMQ?

• RabbitMQ is an open source messaging software that allows you to read messages from and write messages to queues using the Advance Message Queuing Protocol (AMQP).

RabbitMQ是一個使用AMQP協議實現的開源消息隊列，它允許你從隊列中讀取信息和想隊列寫入信息。

• At a basic level an application sends a message to a RabbitMQ server and the server routes that message to a queue. Then another application listening to that queue receives that message and does whatever it needs to with it.

最基本的場景：一個應用將消息發送到RabbitMQ服務，然後服務將消息轉發(route)到指定的隊列。然後，另一個應用監聽這個隊列並且從這個隊列中接受消息並對它進行處理。 

• An application that sends messages is called a producer, and an application reading messages is called a consumer.

發送消息的應用叫做生產者，監聽並消費消息的應用叫做消費者。

• Within a RabbitMQ server it is the exchanges that do the routing of the messages. So a producer will tell the server which exchange it wants to use, and the exchange figures out which queue to put the message on (depicted below).

 在RabbitMQ中，使用的是Exchange來進行消息的轉發。所以生產者會告訴服務器它想使用哪個Exchange，由Exchange來決定向哪個隊列發送消息（如下圖）。

 

• An exchange can have access to many queues. So how does an exchange know which queue(s) to route the message to? Well, there are three types of exchange (direct, fanout and topic) and each type routes messages in a different way. I will use a direct exchange in this article and explain the others later in the series. With a direct exchange, when a queue is declared (created) it is bound to the exchange with a routing key. A routing key is a string such as “blue”, “bob” or “whateveryouwant”. When a producer sends a message it sends a routing key with it. The exchange takes the routing key of the message and matches it the routing keys of the queues bound to it. If any routing keys match, it adds the messages to that queue. If it matches multiple queue’s routing keys, then the messages gets added to them all.

一個Exchange能夠使用很多不同的隊列。那麼它是如何知道改向哪個隊列發送消息的呢？ 實際上，Exchange有3中不同的類型(direct, fanout, topic)並且每種類型都有特定的方法來轉發消息。在本篇文章中，我將會使用direct類型的exchange，其他的exchange類型會在之後解釋。使用direct類型的exchange，當一個隊列被定義(創建)的時刻，它會通過routing key綁定在一個exchange上。routing key 是一個字符串(例如 blue, bob 等等)。Exchange會使用routing key 去匹配綁定在它上面的隊列所指定的routing key。找到任意的routing key之後，Exchange就會將消息發送到這個隊列，如果routing key匹配到了多個隊列，那麼消息就會被髮送到所有匹配到的隊列裏。

• In the example depicted below there are three queues bound to the exchange: A, B and C. Queue A and B are bound to the exchnage with routing key BOB and queue C is bound with routing key BLUE. The producer sends a message with a routing key of BOB and the exchange only puts the message on queues A and B because queue C’s routing key does not match.

下面的例子描述了三個隊列綁定到一個Exchange的情景: A、B、C 三個隊列，隊列A和B使用名爲BOB的routing key綁定到Exchange，隊列C使用名爲BLUE的routing key 綁定到了Exchange。生產者使用BOB這個routing key發送消息時，Exchange只會把消息發送到隊列A和B，因爲隊列C的routing key不匹配。

• Now, as mentioned above, consumers read the messages from queues and in order to do this a consumer must specify with queue it is listening to, or declare a new queue and then listen to that queue. Note that consumers can listen to more than one queue at the same time. So using the example above we can specify a consumer for each queue like so:

現在，就像上面提到的，消費者爲了從隊列中獲取消息，必須指定它所監聽的隊列，或者聲明一個新的隊列並去監聽它。注意：一個消費者能同時監聽多個隊列。示意圖如下：

• Consumer D and Consumer E would receive a message to process. The final thing to say about consumers, for the moment, is that they have to acknowledge that they have processed the message, otherwise RabbitMQ does not know that it can then remove the message.

消費者D個消費者E將會接受消息並處理。最後，關於消費者，要說明的一點是：它們必須確認消息已經被處理完成，否則RabbitMQ無法知道這條消息應不應該移除。

 

Now On To The Code

使用Python3和Kombu來建立一個生產者

• Kombu是一個使用AMQP協議的Python庫。
• 安裝Kombu
  pip install kombu
• 首先，我們需要與RabbitMQ服務建立鏈接。Kombu使用Connection類
  來進行這個操作
  from kombu import Connection
• 然後，我們能夠實例化一個對象並將RabbitMQ的url作爲參數
  rabbit_url = "amqp://localhost:5672/" conn = Connection(rabbit_url)
• 我們需要創建一個信道
  (channnel)來使用連接
  channel = conn.channel()
• 然後，需要指定想要使用的Exchange。Kombu有Exchange類，我們創建了一個direct類型，名叫'example-exchange'的exchange

from kombu import Connection, Exchange 

exchange = Exchange("example-exchange", type="direct")

• 接着是創建生產者。Kombu有Producer類，我們需要傳一個exchange對象和我們創建的channel，同時，還需要指定一個routing key

from kombu import Connection, Exchange, Producer
producer = Producer(exchange=exchange, channel=channel, routing_key="BOB")

• 接下來，需要創建一個用來接收消息的隊列。生產者發送消息並不需要隊列，但是在這個例子中我們還是會創建一個。Kombu有Queue類。我們需要指定隊列的名字，隊列綁定的exchange和routing key

from kombu import Connection, Exchange, Producer, Queue

queuer = Queue(name="example-queue", exchange=exchange, routing_key="BOB")

• 我們需要綁定隊列到exchcange然後再聲明它(此時隊列才真正地創建成功了)

queue.maybe_bind(conn)
queue.declare()

•  最後，我們需要使用producer來發送一條消息

producer.publish("Hello there!")

•  一個完整的生產者就創建完成了，完整代碼如下:

from kombu import Connection, Exchange, Producer, Queue
rabbit_url = “amqp://localhost:5672/”
conn = Connection(rabbit_url)
channel = conn.channel()
exchange = Exchange(“example-exchange”, type=”direct”)
producer = Producer(exchange=exchange, channel=channel, routing_key=”BOB”)
queue = Queue(name=”example-queue”, exchange=exchange, routing_key=”BOB”)
queue.maybe_bind(conn)
queue.declare()
producer.publish(“Hello there!”)

• 如果要檢查隊列是否被創建成功並且隊列中是否存在消息，可以使用RabbitMQ自帶的工具rabbitmqctl：

rabbitmqctl list_queues

• 你會看到下面這樣的輸出

Listing queues …
example-queue 1

創建消費者

• 除了並不需要綁定或者聲明之外(隊列已經在之前創建生產者的時候聲明瞭)，Connection，exchange，和queue的創建方法和之前創建生產者類似。

from kombu import Connection, Exchange, Queue, Consumer
rabbit_url = “amqp://localhost:5672/”
conn = Connection(rabbit_url)
exchange = Exchange(“example-exchange”, type=”direct”)
queue = Queue(name=”example-queue”, exchange=exchange, routing_key=”BOB”)

• 每當消費者接受一個消息的時候，它會調用一個函數來處理這個消息。這被稱作回調函數。我們需要編寫這個回調函數，它有兩個參數: body和message。它只回打印出消息體並確認消息已經被處理了。

def process_message(body, message):
    print(“The body is {}”.format(body))
    message.ack()

• 現在我們需要創建消費者了。它需要一個connection，一個它將要監聽的隊列，回調函數和它接受的消息類型(例如plain text或者json)。消費者是一個上下文管理器因此我們能使用with關鍵字來調用它。

with Consumer(conn, queues=queue, callbacks=[process_message], accept=[“text/plain”]):

• 最後，我們需要可以通過調用drain_events函數來告訴connection去初始化所有的消費者。

conn.drain_events(timeout=2)

• drain_events函數會等待2秒來讓消費者去消費一條消息，當隊列爲空時，會拋出一個異常。如果沒有指定timeout函數，將會一直等待，直到有一條可消費的消息產生。後面會對此有更多介紹。

• 現在一個消費者就創建成功了，完整代碼如下：

from kombu import Connection, Exchange, Queue, Consumer
rabbit_url = “amqp://localhost:5672/”
conn = Connection(rabbit_url)
exchange = Exchange(“example-exchange”, type=”direct”)
queue = Queue(name=”example-queue”, exchange=exchange, routing_key=”BOB”)
def process_message(body, message):
  print(“The body is {}”.format(body))
  message.ack()
with Consumer(conn, queues=queue, callbacks=[process_message], accept=["text/plain"]):  
  conn.drain_events(timeout=2)

• 運行這段代碼，將會產生如下的輸出

The body is Hello there!

現在你可以使用rabbitmqctl來檢測隊列是否爲空了。