Talking RabbitMQ with Python and Kombu

目录

What is RabbitMQ?

Now On To The Code

使用Python3和Kombu来建立一个生产者

创建消费者

---

• 这是一个系列文章里面的的第一篇。主要是关于RabbitMQ的介绍和如何使用Python和Kombu搭建起一个消息队列的系列文章。
• https://medium.com/python-pandemonium/talking-to-rabbitmq-with-python-and-kombu-6cbee93b1298 （需要翻墙）
• 中文翻译：https://www.jianshu.com/p/35a6db9b16f7

 

参考：

深入理解AMQP协议

What is RabbitMQ?

• RabbitMQ is an open source messaging software that allows you to read messages from and write messages to queues using the Advance Message Queuing Protocol (AMQP).

RabbitMQ是一个使用AMQP协议实现的开源消息队列，它允许你从队列中读取信息和想队列写入信息。

• At a basic level an application sends a message to a RabbitMQ server and the server routes that message to a queue. Then another application listening to that queue receives that message and does whatever it needs to with it.

最基本的场景：一个应用将消息发送到RabbitMQ服务，然后服务将消息转发(route)到指定的队列。然后，另一个应用监听这个队列并且从这个队列中接受消息并对它进行处理。 

• An application that sends messages is called a producer, and an application reading messages is called a consumer.

发送消息的应用叫做生产者，监听并消费消息的应用叫做消费者。

• Within a RabbitMQ server it is the exchanges that do the routing of the messages. So a producer will tell the server which exchange it wants to use, and the exchange figures out which queue to put the message on (depicted below).

 在RabbitMQ中，使用的是Exchange来进行消息的转发。所以生产者会告诉服务器它想使用哪个Exchange，由Exchange来决定向哪个队列发送消息（如下图）。

 

• An exchange can have access to many queues. So how does an exchange know which queue(s) to route the message to? Well, there are three types of exchange (direct, fanout and topic) and each type routes messages in a different way. I will use a direct exchange in this article and explain the others later in the series. With a direct exchange, when a queue is declared (created) it is bound to the exchange with a routing key. A routing key is a string such as “blue”, “bob” or “whateveryouwant”. When a producer sends a message it sends a routing key with it. The exchange takes the routing key of the message and matches it the routing keys of the queues bound to it. If any routing keys match, it adds the messages to that queue. If it matches multiple queue’s routing keys, then the messages gets added to them all.

一个Exchange能够使用很多不同的队列。那么它是如何知道改向哪个队列发送消息的呢？ 实际上，Exchange有3中不同的类型(direct, fanout, topic)并且每种类型都有特定的方法来转发消息。在本篇文章中，我将会使用direct类型的exchange，其他的exchange类型会在之后解释。使用direct类型的exchange，当一个队列被定义(创建)的时刻，它会通过routing key绑定在一个exchange上。routing key 是一个字符串(例如 blue, bob 等等)。Exchange会使用routing key 去匹配绑定在它上面的队列所指定的routing key。找到任意的routing key之后，Exchange就会将消息发送到这个队列，如果routing key匹配到了多个队列，那么消息就会被发送到所有匹配到的队列里。

• In the example depicted below there are three queues bound to the exchange: A, B and C. Queue A and B are bound to the exchnage with routing key BOB and queue C is bound with routing key BLUE. The producer sends a message with a routing key of BOB and the exchange only puts the message on queues A and B because queue C’s routing key does not match.

下面的例子描述了三个队列绑定到一个Exchange的情景: A、B、C 三个队列，队列A和B使用名为BOB的routing key绑定到Exchange，队列C使用名为BLUE的routing key 绑定到了Exchange。生产者使用BOB这个routing key发送消息时，Exchange只会把消息发送到队列A和B，因为队列C的routing key不匹配。

• Now, as mentioned above, consumers read the messages from queues and in order to do this a consumer must specify with queue it is listening to, or declare a new queue and then listen to that queue. Note that consumers can listen to more than one queue at the same time. So using the example above we can specify a consumer for each queue like so:

现在，就像上面提到的，消费者为了从队列中获取消息，必须指定它所监听的队列，或者声明一个新的队列并去监听它。注意：一个消费者能同时监听多个队列。示意图如下：

• Consumer D and Consumer E would receive a message to process. The final thing to say about consumers, for the moment, is that they have to acknowledge that they have processed the message, otherwise RabbitMQ does not know that it can then remove the message.

消费者D个消费者E将会接受消息并处理。最后，关于消费者，要说明的一点是：它们必须确认消息已经被处理完成，否则RabbitMQ无法知道这条消息应不应该移除。

 

Now On To The Code

使用Python3和Kombu来建立一个生产者

• Kombu是一个使用AMQP协议的Python库。
• 安装Kombu
  pip install kombu
• 首先，我们需要与RabbitMQ服务建立链接。Kombu使用Connection类
  来进行这个操作
  from kombu import Connection
• 然后，我们能够实例化一个对象并将RabbitMQ的url作为参数
  rabbit_url = "amqp://localhost:5672/" conn = Connection(rabbit_url)
• 我们需要创建一个信道
  (channnel)来使用连接
  channel = conn.channel()
• 然后，需要指定想要使用的Exchange。Kombu有Exchange类，我们创建了一个direct类型，名叫'example-exchange'的exchange

from kombu import Connection, Exchange 

exchange = Exchange("example-exchange", type="direct")

• 接着是创建生产者。Kombu有Producer类，我们需要传一个exchange对象和我们创建的channel，同时，还需要指定一个routing key

from kombu import Connection, Exchange, Producer
producer = Producer(exchange=exchange, channel=channel, routing_key="BOB")

• 接下来，需要创建一个用来接收消息的队列。生产者发送消息并不需要队列，但是在这个例子中我们还是会创建一个。Kombu有Queue类。我们需要指定队列的名字，队列绑定的exchange和routing key

from kombu import Connection, Exchange, Producer, Queue

queuer = Queue(name="example-queue", exchange=exchange, routing_key="BOB")

• 我们需要绑定队列到exchcange然后再声明它(此时队列才真正地创建成功了)

queue.maybe_bind(conn)
queue.declare()

•  最后，我们需要使用producer来发送一条消息

producer.publish("Hello there!")

•  一个完整的生产者就创建完成了，完整代码如下:

from kombu import Connection, Exchange, Producer, Queue
rabbit_url = “amqp://localhost:5672/”
conn = Connection(rabbit_url)
channel = conn.channel()
exchange = Exchange(“example-exchange”, type=”direct”)
producer = Producer(exchange=exchange, channel=channel, routing_key=”BOB”)
queue = Queue(name=”example-queue”, exchange=exchange, routing_key=”BOB”)
queue.maybe_bind(conn)
queue.declare()
producer.publish(“Hello there!”)

• 如果要检查队列是否被创建成功并且队列中是否存在消息，可以使用RabbitMQ自带的工具rabbitmqctl：

rabbitmqctl list_queues

• 你会看到下面这样的输出

Listing queues …
example-queue 1

创建消费者

• 除了并不需要绑定或者声明之外(队列已经在之前创建生产者的时候声明了)，Connection，exchange，和queue的创建方法和之前创建生产者类似。

from kombu import Connection, Exchange, Queue, Consumer
rabbit_url = “amqp://localhost:5672/”
conn = Connection(rabbit_url)
exchange = Exchange(“example-exchange”, type=”direct”)
queue = Queue(name=”example-queue”, exchange=exchange, routing_key=”BOB”)

• 每当消费者接受一个消息的时候，它会调用一个函数来处理这个消息。这被称作回调函数。我们需要编写这个回调函数，它有两个参数: body和message。它只回打印出消息体并确认消息已经被处理了。

def process_message(body, message):
    print(“The body is {}”.format(body))
    message.ack()

• 现在我们需要创建消费者了。它需要一个connection，一个它将要监听的队列，回调函数和它接受的消息类型(例如plain text或者json)。消费者是一个上下文管理器因此我们能使用with关键字来调用它。

with Consumer(conn, queues=queue, callbacks=[process_message], accept=[“text/plain”]):

• 最后，我们需要可以通过调用drain_events函数来告诉connection去初始化所有的消费者。

conn.drain_events(timeout=2)

• drain_events函数会等待2秒来让消费者去消费一条消息，当队列为空时，会抛出一个异常。如果没有指定timeout函数，将会一直等待，直到有一条可消费的消息产生。后面会对此有更多介绍。

• 现在一个消费者就创建成功了，完整代码如下：

from kombu import Connection, Exchange, Queue, Consumer
rabbit_url = “amqp://localhost:5672/”
conn = Connection(rabbit_url)
exchange = Exchange(“example-exchange”, type=”direct”)
queue = Queue(name=”example-queue”, exchange=exchange, routing_key=”BOB”)
def process_message(body, message):
  print(“The body is {}”.format(body))
  message.ack()
with Consumer(conn, queues=queue, callbacks=[process_message], accept=["text/plain"]):  
  conn.drain_events(timeout=2)

• 运行这段代码，将会产生如下的输出

The body is Hello there!

现在你可以使用rabbitmqctl来检测队列是否为空了。