RabbitMQ exchange binding queue原理

RabbitMQ根據exchange的類型規則將消息路由到相應的queue,其中exchange的類型包括direct,topic,fanout以及headers,我們這裏主要介紹topic exchange類型的消息路由原理,其他exchange的路由原理相對簡單。Topic exchange的示意圖如下:

  • *(星號):可以(只能)匹配一個單詞。
  • #(井號):可以匹配多個單詞(或者0個)。

這裏的使用原理不做介紹,主要介紹RabbitMQ源碼內部的實現,即exchange和queue如何進行binding的。

1. exchange與queue的binding

RabbitMQ的exchange和queue的binding代碼流程如下:

對於RabbitMQ的使用流程一般爲:

  • RabbitMQ客戶端與RabbitMQ服務端建立連接。
  • 建立channel。
  • 在建立的channel創建exchange和queue。
  • 將exchange和queue進行binding。
  • 對於生產者而已,向exchange發送消息,且需要指定相應的routing_key。
  • 對於消費者而已,消費處理監聽的queue中的消息。

其中exchange與queue的binding便位於第4個步驟,入口代碼如下:

%% rabbit_channel.erl
%% 處理將交換機exchange和隊列進行綁定的消息
handle_method(#'queue.bind'{queue       = QueueNameBin,
                            exchange    = ExchangeNameBin,
                            routing_key = RoutingKey,
                            nowait      = NoWait,
                            arguments   = Arguments}, _, State) ->
    binding_action(fun rabbit_binding:add/2,
                   ExchangeNameBin, queue, QueueNameBin, RoutingKey, Arguments,
                   #'queue.bind_ok'{}, NoWait, State);

在binding_action函數的一些合法性校驗之後,執行真正的binding函數rabbit_binding:add/2。

%% rabbit_binding.erl
%% 增加新的綁定
add(Binding, InnerFun) ->
    binding_action(
      Binding,
      %% Src,Dst都是從mnesia數據庫中讀取到的數據
      fun (Src, Dst, B) ->
               %% 找到對應交換機的處理模塊去驗證綁定的合法性(Src必須是exchange交換機類型)
               case rabbit_exchange:validate_binding(Src, B) of
                   ok ->
                       %% this argument is used to check queue exclusivity(排他性);
                       %% in general, we want to fail on that in preference to
                       %% anything else
                       %% 功效一:檢查隊列字段exclusive_owner字段的正確性,檢查排他性隊列(如果一個隊列被聲明爲排他隊列,該隊列僅對首次聲明它的連接可見)
                       case InnerFun(Src, Dst) of
                           ok ->
                               %% 查看rabbit_route表中是否有B這個數據,如果沒有則添加到mnesia數據庫表中
                               case mnesia:read({rabbit_route, B}) of
                                   %% 增加綁定的實際操作(實際操作mnesia數據庫表的函數)
                                   []  -> add(Src, Dst, B);
                                   [_] -> fun () -> ok end
                               end;
                           {error, _} = Err ->
                               rabbit_misc:const(Err)
                       end;
                   {error, _} = Err ->
                       rabbit_misc:const(Err)
               end
      end, fun not_found_or_absent_errs/1).

在對應的exchange類型中校驗其合法性,校驗正常後,需要讀取mnesia數據庫中rabbit_route表中數據,查看是否有相應的binding信息,如果有則不再進行binding,沒有則進行binding操作,判斷rabbit_route表數據的原因是:不管exchange和queue是否需要持久化,最後都會將其binding信息寫入到rabbit_route表中。

這裏提一下,RabbitMQ所保存的一些metadata信息依賴於mnesia分佈式數據庫,其中RabbitMQ所使用的表可以通過以下命令進行顯示:

[root@master scripts]# ./rabbitmqctl eval 'mnesia:info().'
……
===> System info in version "4.16", debug level = none <===
opt_disc. Directory "/var/lib/rabbitmq/mnesia/rabbit@master" is used.
use fallback at restart = false
running db nodes   = [rabbit@master]
stopped db nodes   = []
master node tables = []
remote             = []
ram_copies         = [gm_group,mirrored_sup_childspec,rabbit_exchange,
                      rabbit_exchange_serial,
                      rabbit_exchange_type_consistent_hash,
                      rabbit_exchange_type_consistent_hash_ring_state,
                      rabbit_listener,rabbit_queue,rabbit_reverse_route,
                      rabbit_route,rabbit_semi_durable_route,
                      rabbit_topic_trie_binding,rabbit_topic_trie_edge,
                      rabbit_topic_trie_node,rh_exchange_table,
                      tracked_connection_on_node_rabbit@master,
                      tracked_connection_per_vhost_on_node_rabbit@master,
                      x_jms_topic_table]
disc_copies        = [rabbit_durable_exchange,rabbit_durable_queue,
                      rabbit_durable_route,rabbit_runtime_parameters,
                      rabbit_topic_permission,rabbit_user,
                      rabbit_user_permission,rabbit_vhost,schema]
disc_only_copies   = []
[{rabbit@master,disc_copies}] = [rabbit_runtime_parameters,
                                 rabbit_durable_exchange,rabbit_durable_queue,
                                 rabbit_user,rabbit_durable_route,
                                 rabbit_topic_permission,rabbit_vhost,schema,
                                 rabbit_user_permission]
[{rabbit@master,ram_copies}] = [rabbit_topic_trie_node,
                                tracked_connection_per_vhost_on_node_rabbit@master,
                                x_jms_topic_table,
                                rabbit_exchange_type_consistent_hash_ring_state,
                                rabbit_reverse_route,
                                rabbit_topic_trie_binding,
                                rabbit_exchange_type_consistent_hash,gm_group,
                                rabbit_listener,mirrored_sup_childspec,
                                rabbit_exchange,rabbit_route,
                                rabbit_exchange_serial,
                                tracked_connection_on_node_rabbit@master,
                                rabbit_semi_durable_route,rh_exchange_table,
                                rabbit_queue,rabbit_topic_trie_edge]
21 transactions committed, 7 aborted, 0 restarted, 0 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
ok

其中disc_copies表示會將數據存入磁盤和內存,ram_copies表示將數據存入內存。disc_only_copies表示將數據只存入磁盤。由於mnesia底層是基於ets和dets實現,所以可以通過ets或者dets查看錶中數據,注意ets和dets是隻能查詢本節點的表數據信息。如查看rabbit_exchang表數據信息:

[root@master scripts]# ./rabbitmqctl eval 'ets:tab2list(rabbit_exchange).'
[{exchange,{resource,<<"/">>,exchange,<<"amq.direct">>},
           direct,true,false,false,[],undefined,undefined,undefined,
           {[],[rabbit_event_exchange_decorator]},
           #{}},
……
{exchange,{resource,<<"/">>,exchange,<<"amq.headers">>},
           headers,true,false,false,[],undefined,undefined,undefined,
           {[],[rabbit_event_exchange_decorator]},
           #{}}]

具體參看erlang官方文檔的mnesia,ets和dets的使用說明。繼續回到binding操作,假設第一次進行binding操作,則將執行下列操作:

%% rabbit_binding.erl
%% 增加綁定的實際操作(實際操作mnesia數據庫表的函數)
add(Src, Dst, B) ->
    [SrcDurable, DstDurable] = [durable(E) || E <- [Src, Dst]],
    case (SrcDurable andalso DstDurable andalso
              mnesia:read({rabbit_durable_route, B}) =/= []) of
        false -> %% 根據交換機exchange和隊列的持久化狀態,將路由信息寫入mnesia數據庫表
                 ok = sync_route(#route{binding = B}, SrcDurable, DstDurable,
                                 fun mnesia:write/3),
                 %% 交換機exchange創建後的回調
                 x_callback(transaction, Src, add_binding, B),
                 %% 讓Src對應的exchange交換機類型對應的模塊和所有的修飾模塊回調serialise_events函數,如果執行成功,則將XName在rabbit_exchange_serial表中的next值加一
                 Serial = rabbit_exchange:serial(Src),
                 fun () ->
                          x_callback(Serial, Src, add_binding, B),
                          %% 向rabbit_event事件中心發佈綁定信息被創建的事件
                          ok = rabbit_event:notify(binding_created, info(B))
                 end;
        true  -> rabbit_misc:const({error, binding_not_found})
    end.

sync_route函數將exchange和queue的binding信息根據exchange和queue是否持久化來判斷是否寫入rabbit_durable_route和rabbit_semi_durable_route表中,但無論是否持久化都會寫入rabbit_route和rabbit_reverse_route表中。即

  • rabbit_durable_route表:exchange和queue都持久化則寫入該表。
  • rabbit_semi_durable_route表:exchange不需要持久化,queue需要持久化則寫入該表。
  • rabbit_route和rabbit_reverse_route表:exchange和queue是否持久化都會寫入該表。

除了binding信息寫入上述表以外,對於類型爲topic的exchange,RabbitMQ還會將binding信息寫入rabbit_topic_trie_node,rabbit_topic_trie_edge和rabbit_topic_trie_binding表中。

%% rabbit_exchange_type_topic.erl
%% 路由綁定信息的添加回調該模塊進行相關的處理(transaction:事務)
add_binding(transaction, _Exchange, Binding) ->
    internal_add_binding(Binding);

%% rabbit_exchange_type_topic.erl
%% exchange交換機內部添加綁定信息的接口
internal_add_binding(#binding{source = X, key = K, destination = D,
                              args = Args}) ->
    %% 創建新的節點和邊
    FinalNode = follow_down_create(X, split_topic_key(K)),
    %% 創建綁定信息
    trie_add_binding(X, FinalNode, D, Args),
    ok.

其內部原理可參考以下鏈接:https://www.erlang-solutions.com/blog/rabbit-s-anatomy-understanding-topic-exchanges.html

再提一下,我們執行rabbitmqctl list_bindings命令時,查看到的exchange和queue的bindings信息是從rabbit_route表中進行獲取的。即

%% rabbit_binding.erl
%% 列出VHostPath下的所有路由綁定信息
list(VHostPath) ->
    VHostResource = rabbit_misc:r(VHostPath, '_'),
    Route = #route{binding = #binding{source      = VHostResource,
                                      destination = VHostResource,
                                      _           = '_'},
                   _       = '_'},
    [B || #route{binding = B} <- mnesia:dirty_match_object(rabbit_route,
                                                           Route)].

2. 總結

1 RabbitMQ的exchange和queue的binding的建立在客戶端與RabbitMQ之間建立連接且exchange和queue創建完成後。

2 binding信息會被保存到mnesia數據庫中,其保存的表爲rabbit_route表,根據exchange和queue是否持久化判斷是否保存到rabbit_durable_route和rabbit_semi_durable_route表中。

3 對於topic類型的exchange,其binding信息還會被保存到rabbit_topic_trie_node,rabbit_topic_trie_edge和rabbit_topic_trie_binding表中。

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章