RabbitMQ(消息隊列)

線程queue：只能是同一個進程下的線程進行消息交互
進程queue：只能是父進程和子進程或者是同一進程下的子進程進行信息交互

RabiitMQ是中間代理，它可以實現不同語言之間、不同獨立進程之間、不同機器之間的消息交互
工作原理：

實現qq和word之間的通信，如上圖，可以有三種方式，
第一種方式效率低下，
第二種方式重新開發一條通道，成本太高，
第三種方式，中間代理的方法，比如RabbitMQ、ZeroMQ、ActiveMQ等。

安裝rabbitMQ：brew install rabbitmq
cd /usr/local/sbin
啓動rabbitmq服務器：sudo ./rabbitmq-server
查看rabbitmq中有哪些消息隊列：sudo ./rabbitmqctl list_queues

安裝python rabbitMQ moudle：pip3 install pika

1、實現最簡單的隊列通信

send端

# Author: 73

import pika

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost"))

channel = connection.channel() # 聲明一個管道

channel.queue_declare(queue='hello') # 聲明queue

# RabbitMQ a message can never be sent directly to the queue, it always needs to go through an exchange.
channel.basic_publish(
    exchange='',
    routing_key="hello",
    body="hello world!")

print("send done...")

connection.close()

receive端

# Author: 73

import pika

conn = pika.BlockingConnection(pika.ConnectionParameters("localhost"))

channel = conn.channel()

#You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
#was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
channel.queue_declare(queue="hello")

def callback(ch, method, properties, body):
    print("received message: ", body)

channel.basic_consume(queue="hello",
                      on_message_callback=callback,
                      auto_ack=True)

print("waiting for message...")
channel.start_consuming()

遠程連接rabbitmq server的話，需要配置權限

首先在rabbitmq server上創建一個用戶

sudo rabbitmqctl  add_user seth seth1234

同時還要配置權限，允許從外面訪問

sudo rabbitmqctl set_permissions -p / seth ".*" ".*" ".*"

	set_permissions [-p vhost] {user} {conf} {write} {read}
	
	vhost
	The name of the virtual host to which to grant the user access, defaulting to /.
	
	user
	The name of the user to grant access to the specified virtual host.
	
	conf
	A regular expression matching resource names for which the user is granted configure permissions.
	
	write
	A regular expression matching resource names for which the user is granted write permissions.
	
	read
	A regular expression matching resource names for which the user is granted read permissions.

客戶端連接的時候需要配置認證參數

credentials=pika.PlainCredentials('alex', 'alex3714')

connection=pika.BlockingConnection(pika.ConnectionParameters('10.211.55.5',5672,'/',credentials))

channel=connection.channel()

2、消息分發輪詢

還是上面的兩個程序，現在依次啓動三個receive端，然後依次啓動4次send端，運行結果是第一個啓動的receive端先收到第一條消息，然後第二個receive端收到第二條消息，第三個receive端收到第三條消息，最後第一個receive端收到了第4條消息。這就是rabbitmq的消息分發輪詢。

3、receive端消息處理完畢響應

如果receive端在處理消息的時候，突然，崩了，那麼這條消息會怎麼樣呢？
答：如果有別的receive還開着，就會被別的receive接收；如果沒有別的receive，那這條消息就會還在內存中的queue裏面，所以我們需要消息處理完畢之後，給服務器發出響應，消息處理完畢，將它從queue裏面移除。

修改後的receive端

# Author: 73

import pika, time

conn = pika.BlockingConnection(pika.ConnectionParameters("localhost"))

channel = conn.channel()

#You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
#was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
channel.queue_declare(queue="hello")

def callback(ch, method, properties, body):
    print(ch, method, properties)
    time.sleep(30)
    print("received message: ", body)
    ch.basic_ack(delivery_tag=method.delivery_tag) # 手動給服務端確認消息已被處理完畢

channel.basic_consume(queue="hello",
                      on_message_callback=callback,
                      auto_ack=False # 給服務端確認消息是否被處理
                      )

print("waiting for message...")
channel.start_consuming()

4、消息持久化

那如果是服務器死機了，那麼還沒有被處理的消息會怎麼樣呢？
答：消息會徹底丟失

這種風險我們必須避免，那怎麼做？

首先，爲了避免rabbitmq不會丟失queue，我們需要聲明它爲durable

channel.queue_declare(queue='hello', durable=True)

注：RabbitMQ doesn’t allow you to redefine an existing queue with different parameters and will return an error to any program that tries to do that.

到這裏，我們只是持久化了隊列，保證rabbitmq在重啓或宕機的情況下，隊列不會丟失，但是隊列內的數據卻不會保存下來，所以，我們還需要聲明數據持久化

channel.basic_publish(
    exchange='',
    routing_key="hello",
    body="hello world!",
    properties=pika.BasicProperties(
        delivery_mode=2,  # make message persistent
    ))

5、消息公平分發

如果Rabbit只管按順序把消息發到各個消費者身上，不考慮消費者負載的話，很可能出現，一個機器配置不高的消費者那裏堆積了很多消息處理不完，同時配置高的消費者卻一直很輕鬆。爲解決此問題，可以在各個消費者端，配置perfetch=1,意思就是告訴RabbitMQ在我這個消費者當前消息還沒處理完的時候就不要再給我發新消息了。

channel.basic_qos(prefetch_count=1)

帶消息持久化+公平分發的完整代碼

send端

# Author: 73

import pika

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost"))

channel = connection.channel() # 聲明一個管道

channel.queue_declare(queue='hello', durable=True) # 聲明queue

# RabbitMQ a message can never be sent directly to the queue, it always needs to go through an exchange.
channel.basic_publish(
    exchange='',
    routing_key="hello",
    body="hello world!",
    properties=pika.BasicProperties(
        delivery_mode=2,  # make message persistent
    ))

print("send done...")

connection.close()

receive1端

# Author: 73

import pika, time

conn = pika.BlockingConnection(pika.ConnectionParameters("localhost"))

channel = conn.channel()

#You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
#was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
channel.queue_declare(queue="hello", durable=True)

def callback(ch, method, properties, body):
    print(ch, method, properties)
    #time.sleep(30)
    print("received message: ", body)
    ch.basic_ack(delivery_tag=method.delivery_tag) # 手動給服務端確認消息已被處理完畢

channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue="hello",
                      on_message_callback=callback,
                      auto_ack=False, # 給服務端確認消息是否被處理
                      )

print("waiting for message...")
channel.start_consuming()

receive2端

# Author: 73

import pika, time

conn = pika.BlockingConnection(pika.ConnectionParameters("localhost"))

channel = conn.channel()

#You may ask why we declare the queue again ‒ we have already declared it in our previous code.
# We could avoid that if we were sure that the queue already exists. For example if send.py program
#was run before. But we're not yet sure which program to run first. In such cases it's a good
# practice to repeat declaring the queue in both programs.
channel.queue_declare(queue="hello", durable=True)

def callback(ch, method, properties, body):
    print(ch, method, properties)
    time.sleep(30)
    print("received message: ", body)
    ch.basic_ack(delivery_tag=method.delivery_tag) # 手動給服務端確認消息已被處理完畢

channel.basic_qos(prefetch_count=1)
channel.basic_consume(queue="hello",
                      on_message_callback=callback,
                      auto_ack=False, # 給服務端確認消息是否被處理
                      )

print("waiting for message...")
channel.start_consuming()

6、Publish\Subscribe(消息發佈\訂閱)

前面的都是1對1的消息發送和接收，即消息只能發送到指定的queue裏，但有些時候你想讓你的消息被所有的Queue收到，類似廣播的效果，這時候就要用到exchange了

羣發消息（(exchange type=fanout)）
所有bind到此exchange的queue都可以接收消息

publisher

# Author: 73

import sys, pika

conn = pika.BlockingConnection(
    pika.ConnectionParameters(
        host="localhost"
    )
)

channel = conn.channel()

channel.exchange_declare(exchange="logs", exchange_type="fanout")

message = " ".join(sys.argv[1:]) or "hello world..."

channel.basic_publish(
    exchange="logs",
    routing_key="",
    body=message,
)

print('send done...')

conn.close()

consume

# Author: 73

import pika

conn = pika.BlockingConnection(
    pika.ConnectionParameters(
        host="localhost",
    )
)

chanel = conn.channel()

chanel.exchange_declare(exchange="logs", exchange_type="fanout")

result = chanel.queue_declare('', exclusive=True) #不指定queue名字,rabbit會隨機分配一個名字,exclusive=True會在使用此queue的消費者斷開後,自動將queue刪除

queue_name = result.method.queue

chanel.queue_bind(exchange="logs", queue=queue_name)

def callback(ch, method, properties, body):
    print("recv: ", body)
    ch.basic_ack(delivery_tag=method.delivery_tag)  # 手動給服務端確認消息已被處理完畢

chanel.basic_consume(
    queue=queue_name,
    on_message_callback=callback,
    auto_ack=False
)

chanel.start_consuming()

有選擇的接收消息(exchange type=direct)
RabbitMQ還支持根據關鍵字發送，即：隊列綁定關鍵字，發送者將數據根據關鍵字發送到消息exchange，exchange根據關鍵字判定應該將數據發送至指定隊列。
通過routingKey和exchange決定的那個唯一的queue可以接收消息

publish

# Author: 73

import pika, sys

conn = pika.BlockingConnection(
    pika.ConnectionParameters(
        host="localhost",
    )
)
channel = conn.channel()
channel.exchange_declare(
    exchange="direct_logs",
    exchange_type="direct"
)
severity = sys.argv[1] if len(sys.argv)>1 else "info"
message = ' '.join(sys.argv[2:]) or "hello world"
channel.basic_publish(
    exchange="direct_logs",
    routing_key=severity,
    body=message
)
print('[%s] send done...' % message)
conn.close()

consume

# Author: 73

import pika, sys

conn = pika.BlockingConnection(pika.ConnectionParameters(host="localhost"))

channel = conn.channel()

channel.exchange_declare(
    exchange="direct_logs",
    exchange_type="direct"
)

result = channel.queue_declare('', exclusive=True)
queue_name = result.method.queue

severities = sys.argv[1:]
if not severities:
    print("%s must have [info] [error] [debug]" % sys.argv[0])
    sys.exit(0)

for severity in severities:
    channel.queue_bind(
        exchange="direct_logs",
        queue=queue_name,
        routing_key=severity,
    )

def callback(ch, method, properity, body):
    print("recv: ", body)

channel.basic_consume(
    queue=queue_name,
    on_message_callback=callback,
    auto_ack=False
)
channel.start_consuming()

運行結果：

更細緻的消息過濾（exchange type=topic）

使用RoutingKey爲#，Exchange Type爲topic的時候相當於使用fanout
#代表一個或多個字符，*代表任何字符
例：#.a會匹配a.a，aa.a，aaa.a等
*.a會匹配a.a，b.a，c.a等

publisher

# Author: 73

import pika, sys

conn = pika.BlockingConnection(
    pika.ConnectionParameters(
        host="localhost",
    )
)
channel = conn.channel()
channel.exchange_declare(
    exchange="topic_logs",
    exchange_type="topic"
)
routing_key = sys.argv[1] if len(sys.argv)>1 else "qq.info"
message = ' '.join(sys.argv[2:]) or "hello world"
channel.basic_publish(
    exchange="topic_logs",
    routing_key=routing_key,
    body=message
)
print('[%s] send done...' % message)
conn.close()

consume

# Author: 73

import pika, sys

conn = pika.BlockingConnection(pika.ConnectionParameters(host="localhost"))

channel = conn.channel()

channel.exchange_declare(
    exchange="topic_logs",
    exchange_type="topic"
)

result = channel.queue_declare('', exclusive=True)
queue_name = result.method.queue

binding_keys = sys.argv[1:]
if not binding_keys:
    print("%s must have [info] [error] [debug]" % sys.argv[0])
    sys.exit(0)

for binding_key in binding_keys:
    channel.queue_bind(
        exchange="topic_logs",
        queue=queue_name,
        routing_key=binding_key,
    )

def callback(ch, method, properity, body):
    print("recv: ", body)

channel.basic_consume(
    queue=queue_name,
    on_message_callback=callback,
    auto_ack=False
)
channel.start_consuming()

python基礎10_1-RabbitMQ(消息隊列)

目錄

RabbitMQ(消息隊列)

1、實現最簡單的隊列通信

2、消息分發輪詢

3、receive端消息處理完畢響應

4、消息持久化

5、消息公平分發

6、Publish\Subscribe(消息發佈\訂閱)

vex edr v5之軸角編碼器和超聲波傳感器

python基礎2_1-列表、元組、字符串、字典、集合

css-後臺頁面管理佈局和圖標的使用

python基礎4_1-裝飾器(decorator)

JavaScript跑馬燈實例

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結