RabbitMQ(消息队列)

线程queue：只能是同一个进程下的线程进行消息交互
进程queue：只能是父进程和子进程或者是同一进程下的子进程进行信息交互

RabiitMQ是中间代理，它可以实现不同语言之间、不同独立进程之间、不同机器之间的消息交互
工作原理：

实现qq和word之间的通信，如上图，可以有三种方式，
第一种方式效率低下，
第二种方式重新开发一条通道，成本太高，
第三种方式，中间代理的方法，比如RabbitMQ、ZeroMQ、ActiveMQ等。

安装rabbitMQ：brew install rabbitmq
cd /usr/local/sbin
启动rabbitmq服务器：sudo ./rabbitmq-server
查看rabbitmq中有哪些消息队列：sudo ./rabbitmqctl list_queues

安装python rabbitMQ moudle：pip3 install pika

1、实现最简单的队列通信

send端

# Author: 73

import pika

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost"))

channel = connection.channel() # 声明一个管道

channel.queue_declare(queue='hello') # 声明queue

# RabbitMQ a message can never be sent directly to the queue, it always needs to go through an exchange.
channel.basic_publish(
    exchange='',
    routing_key="hello",
    body="hello world!")

print("send done...")

connection.close()

receive端

# Author: 73

import pika

conn = pika.BlockingConnection(pika.ConnectionParameters("localhost"))

channel = conn.channel()

#You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
#was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
channel.queue_declare(queue="hello")

def callback(ch, method, properties, body):
    print("received message: ", body)

channel.basic_consume(queue="hello",
                      on_message_callback=callback,
                      auto_ack=True)

print("waiting for message...")
channel.start_consuming()

远程连接rabbitmq server的话，需要配置权限

首先在rabbitmq server上创建一个用户

sudo rabbitmqctl  add_user seth seth1234

同时还要配置权限，允许从外面访问

sudo rabbitmqctl set_permissions -p / seth ".*" ".*" ".*"

	set_permissions [-p vhost] {user} {conf} {write} {read}
	
	vhost
	The name of the virtual host to which to grant the user access, defaulting to /.
	
	user
	The name of the user to grant access to the specified virtual host.
	
	conf
	A regular expression matching resource names for which the user is granted configure permissions.
	
	write
	A regular expression matching resource names for which the user is granted write permissions.
	
	read
	A regular expression matching resource names for which the user is granted read permissions.

客户端连接的时候需要配置认证参数

credentials=pika.PlainCredentials('alex', 'alex3714')

connection=pika.BlockingConnection(pika.ConnectionParameters('10.211.55.5',5672,'/',credentials))

channel=connection.channel()

2、消息分发轮询

还是上面的两个程序，现在依次启动三个receive端，然后依次启动4次send端，运行结果是第一个启动的receive端先收到第一条消息，然后第二个receive端收到第二条消息，第三个receive端收到第三条消息，最后第一个receive端收到了第4条消息。这就是rabbitmq的消息分发轮询。

3、receive端消息处理完毕响应

如果receive端在处理消息的时候，突然，崩了，那么这条消息会怎么样呢？
答：如果有别的receive还开着，就会被别的receive接收；如果没有别的receive，那这条消息就会还在内存中的queue里面，所以我们需要消息处理完毕之后，给服务器发出响应，消息处理完毕，将它从queue里面移除。

修改后的receive端

# Author: 73

import pika, time

conn = pika.BlockingConnection(pika.ConnectionParameters("localhost"))

channel = conn.channel()

#You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
#was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
channel.queue_declare(queue="hello")

def callback(ch, method, properties, body):
    print(ch, method, properties)
    time.sleep(30)
    print("received message: ", body)
    ch.basic_ack(delivery_tag=method.delivery_tag) # 手动给服务端确认消息已被处理完毕

channel.basic_consume(queue="hello",
                      on_message_callback=callback,
                      auto_ack=False # 给服务端确认消息是否被处理
                      )

print("waiting for message...")
channel.start_consuming()

4、消息持久化

那如果是服务器死机了，那么还没有被处理的消息会怎么样呢？
答：消息会彻底丢失

这种风险我们必须避免，那怎么做？

首先，为了避免rabbitmq不会丢失queue，我们需要声明它为durable

channel.queue_declare(queue='hello', durable=True)

注：RabbitMQ doesn’t allow you to redefine an existing queue with different parameters and will return an error to any program that tries to do that.

到这里，我们只是持久化了队列，保证rabbitmq在重启或宕机的情况下，队列不会丢失，但是队列内的数据却不会保存下来，所以，我们还需要声明数据持久化

channel.basic_publish(
    exchange='',
    routing_key="hello",
    body="hello world!",
    properties=pika.BasicProperties(
        delivery_mode=2,  # make message persistent
    ))

5、消息公平分发

如果Rabbit只管按顺序把消息发到各个消费者身上，不考虑消费者负载的话，很可能出现，一个机器配置不高的消费者那里堆积了很多消息处理不完，同时配置高的消费者却一直很轻松。为解决此问题，可以在各个消费者端，配置perfetch=1,意思就是告诉RabbitMQ在我这个消费者当前消息还没处理完的时候就不要再给我发新消息了。

channel.basic_qos(prefetch_count=1)

带消息持久化+公平分发的完整代码

send端

# Author: 73

import pika

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost"))

channel = connection.channel() # 声明一个管道

channel.queue_declare(queue='hello', durable=True) # 声明queue

# RabbitMQ a message can never be sent directly to the queue, it always needs to go through an exchange.
channel.basic_publish(
    exchange='',
    routing_key="hello",
    body="hello world!",
    properties=pika.BasicProperties(
        delivery_mode=2,  # make message persistent
    ))

print("send done...")

connection.close()

receive1端

# Author: 73

import pika, time

conn = pika.BlockingConnection(pika.ConnectionParameters("localhost"))

channel = conn.channel()

#You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
#was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
channel.queue_declare(queue="hello", durable=True)

def callback(ch, method, properties, body):
    print(ch, method, properties)
    #time.sleep(30)
    print("received message: ", body)
    ch.basic_ack(delivery_tag=method.delivery_tag) # 手动给服务端确认消息已被处理完毕

channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue="hello",
                      on_message_callback=callback,
                      auto_ack=False, # 给服务端确认消息是否被处理
                      )

print("waiting for message...")
channel.start_consuming()

receive2端

# Author: 73

import pika, time

conn = pika.BlockingConnection(pika.ConnectionParameters("localhost"))

channel = conn.channel()

#You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
#was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
channel.queue_declare(queue="hello", durable=True)

def callback(ch, method, properties, body):
    print(ch, method, properties)
    time.sleep(30)
    print("received message: ", body)
    ch.basic_ack(delivery_tag=method.delivery_tag) # 手动给服务端确认消息已被处理完毕

channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue="hello",
                      on_message_callback=callback,
                      auto_ack=False, # 给服务端确认消息是否被处理
                      )

print("waiting for message...")
channel.start_consuming()

6、Publish\Subscribe(消息发布\订阅)

前面的都是1对1的消息发送和接收，即消息只能发送到指定的queue里，但有些时候你想让你的消息被所有的Queue收到，类似广播的效果，这时候就要用到exchange了

群发消息（(exchange type=fanout)）
所有bind到此exchange的queue都可以接收消息

publisher

# Author: 73

import sys, pika

conn = pika.BlockingConnection(
    pika.ConnectionParameters(
        host="localhost"
    )
)

channel = conn.channel()

channel.exchange_declare(exchange="logs", exchange_type="fanout")

message = " ".join(sys.argv[1:]) or "hello world..."

channel.basic_publish(
    exchange="logs",
    routing_key="",
    body=message,
)

print('send done...')

conn.close()

consume

# Author: 73

import pika

conn = pika.BlockingConnection(
    pika.ConnectionParameters(
        host="localhost",
    )
)

chanel = conn.channel()

chanel.exchange_declare(exchange="logs", exchange_type="fanout")

result = chanel.queue_declare('', exclusive=True) #不指定queue名字,rabbit会随机分配一个名字,exclusive=True会在使用此queue的消费者断开后,自动将queue删除

queue_name = result.method.queue

chanel.queue_bind(exchange="logs", queue=queue_name)

def callback(ch, method, properties, body):
    print("recv: ", body)
    ch.basic_ack(delivery_tag=method.delivery_tag)  # 手动给服务端确认消息已被处理完毕

chanel.basic_consume(
    queue=queue_name,
    on_message_callback=callback,
    auto_ack=False
)

chanel.start_consuming()

有选择的接收消息(exchange type=direct)
RabbitMQ还支持根据关键字发送，即：队列绑定关键字，发送者将数据根据关键字发送到消息exchange，exchange根据关键字判定应该将数据发送至指定队列。
通过routingKey和exchange决定的那个唯一的queue可以接收消息

publish

# Author: 73

import pika, sys

conn = pika.BlockingConnection(
    pika.ConnectionParameters(
        host="localhost",
    )
)
channel = conn.channel()
channel.exchange_declare(
    exchange="direct_logs",
    exchange_type="direct"
)
severity = sys.argv[1] if len(sys.argv)>1 else "info"
message = ' '.join(sys.argv[2:]) or "hello world"
channel.basic_publish(
    exchange="direct_logs",
    routing_key=severity,
    body=message
)
print('[%s] send done...' % message)
conn.close()

consume

# Author: 73

import pika, sys

conn = pika.BlockingConnection(pika.ConnectionParameters(host="localhost"))

channel = conn.channel()

channel.exchange_declare(
    exchange="direct_logs",
    exchange_type="direct"
)

result = channel.queue_declare('', exclusive=True)
queue_name = result.method.queue

severities = sys.argv[1:]
if not severities:
    print("%s must have [info] [error] [debug]" % sys.argv[0])
    sys.exit(0)

for severity in severities:
    channel.queue_bind(
        exchange="direct_logs",
        queue=queue_name,
        routing_key=severity,
    )

def callback(ch, method, properity, body):
    print("recv: ", body)

channel.basic_consume(
    queue=queue_name,
    on_message_callback=callback,
    auto_ack=False
)
channel.start_consuming()

运行结果：

更细致的消息过滤（exchange type=topic）

使用RoutingKey为#，Exchange Type为topic的时候相当于使用fanout
#代表一个或多个字符，*代表任何字符
例：#.a会匹配a.a，aa.a，aaa.a等
*.a会匹配a.a，b.a，c.a等

publisher

# Author: 73

import pika, sys

conn = pika.BlockingConnection(
    pika.ConnectionParameters(
        host="localhost",
    )
)
channel = conn.channel()
channel.exchange_declare(
    exchange="topic_logs",
    exchange_type="topic"
)
routing_key = sys.argv[1] if len(sys.argv)>1 else "qq.info"
message = ' '.join(sys.argv[2:]) or "hello world"
channel.basic_publish(
    exchange="topic_logs",
    routing_key=routing_key,
    body=message
)
print('[%s] send done...' % message)
conn.close()

consume

# Author: 73

import pika, sys

conn = pika.BlockingConnection(pika.ConnectionParameters(host="localhost"))

channel = conn.channel()

channel.exchange_declare(
    exchange="topic_logs",
    exchange_type="topic"
)

result = channel.queue_declare('', exclusive=True)
queue_name = result.method.queue

binding_keys = sys.argv[1:]
if not binding_keys:
    print("%s must have [info] [error] [debug]" % sys.argv[0])
    sys.exit(0)

for binding_key in binding_keys:
    channel.queue_bind(
        exchange="topic_logs",
        queue=queue_name,
        routing_key=binding_key,
    )

def callback(ch, method, properity, body):
    print("recv: ", body)

channel.basic_consume(
    queue=queue_name,
    on_message_callback=callback,
    auto_ack=False
)
channel.start_consuming()

python基础10_1-RabbitMQ(消息队列)

目录

RabbitMQ(消息队列)

1、实现最简单的队列通信

2、消息分发轮询

3、receive端消息处理完毕响应

4、消息持久化

5、消息公平分发

6、Publish\Subscribe(消息发布\订阅)

vex edr v5之軸角編碼器和超聲波傳感器

python基礎2_1-列表、元組、字符串、字典、集合

css-後臺頁面管理佈局和圖標的使用

python基礎4_1-裝飾器(decorator)

JavaScript跑馬燈實例

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結