ActiveMQ 生產者流量控制(Producer Flow Control)

最近學習ActiveMq,昨日查看其配置文件activemq.xml的時候,被一行註釋引到了http://activemq.apache.org/producer-flow-control.html頁面,感覺挺有用,遂翻譯之保存成文。作爲自己的第一篇譯作,必有諸多不妥之處,故附上原文作對照,歡迎拍磚。


Producer Flow Control生產者流量控制

In ActiveMQ 4.x flow control was implemented using TCP flow control. The underlying network connection of throttled consumers was suspended to enforce flow control limits. This strategy is very efficient but can lead to deadlocks if there are multiple producers and consumers sharing the same connection.

在ActiveMQ 4.x版本中,流量控制是用TCP流量控制實現的。受節制的消費者底層的網絡連接被掛起,以強制進行流量控制限制。這個策略非常高效,但是如果有多個生產者和消費者共享同一個連接的時候,可能會導致死鎖。

As of ActiveMQ 5.0, we can now individually flow control each producer on a shared connection without having to suspend the entire connection. By 'flow control' we mean that if the broker detects that the memory limit for the destination, or the temp- or file-store limits for the broker, have been exceeded, then the flow of messages can be slowed down. The producer will be either blocked until resources are available or will receive a JMSException: this behaviour is configurable and described in the section below on <systemUsage>.

 在ActiveMQ5.0版本中,我們可以分別對一個共享連接上的各個生產者進行流量控制,而不需要掛起整個連接。“流量控制”意味着當代理(broker)檢測到目標(destination)的內存,或temp-或file-store超過了限制,消息的流量可以被減慢。生產者將會被阻塞直至資源可用,或者收到一個JMSException異常:這種行爲是可配置的,下面的<systemUsage>章節會描述到。

It's worth noting that the default <systemUsage> settings will cause the producer to block when the memoryLimit or <systemUsage> limits are reached: this blocking behaviour is sometimes misinterpreted as a 'hung producer', when in fact the producer is simply diligently waiting until space is available.

值得注意的是,當內存限制或<systemUsage>限制達到的時候,<systemUsage>默認的設置會引起生產值阻塞:這種阻塞行爲有時會被誤解爲“掛起的生產者”,而事實是生產者只是勤奮地等着空間可用。

  • Messages that are sent synchronously will automatically use per producer flow control; this applies generally to persistent messages which are sent synchronously unless you enable the useAsyncSend flag.
  • 同步發送的消息將會自動對每一個生產者使用流量控制;除非你使能了useAsyncSend標誌,否則這將對同步發送的持久化消息都適用。
  • Producers that use Async Sends - generally speaking, producers of non-persistent messages - don't bother waiting for any acknowledgement from the broker; so, if a memory limit has been exceeded, you will not get notfied. If you do want to be aware of broker limits being exceeded, you will need to configure the ProducerWindowSize connection option so that even async messages are flow controlled per producer.
  • 使用異步發送的生產者 —— 一般來說,就是發送非持久化消息的生產者 —— 不需要等候來自代理的任何確認消息;所以,如果內存限制被超過了,你不會被通知。如果你真的想什麼時候代理的限制被超過了,你需要配置ProducerWindowSize這一連接選項,這樣就算是異步消息也會對每一個生產者進行流量控制。
  1. ActiveMQConnectionFactory connctionFactory = ...
  2. connctionFactory.setProducerWindowSize(1024000);

The ProducerWindowSize is the maximum number of bytes of data that a producer will transmit to a broker before waiting for acknowledgment messages from the broker that it has accepted the previously sent messages.

ProducerWindowSize是一個生產者在等到確認消息之前,可以發送給代理的數據的最大byte數,這個確認消息用來告訴生產者,代理已經收到先前發送的消息了。

Alternatively, if you're sending non-persisted messages (which are by default sent async), and want to be informed if the queue or topic's memory limit has been breached, then you can simply configure the connection factory to 'alwaysSyncSend'. While this is going to be slower, it will ensure that your message producer is informed immediately of memory issues.

或者,如果你要發送非持久化的消息(該消息默認是異步發送的),並且想要得到隊列或者主題的內存限制是否達到,你只需將連接工廠配置爲“alwaysSyncSend”。雖然這樣會變得稍微慢一點,但是這將保證當出現內存問題時,你的消息生產者能夠及時得到通知。

If you like, you can disable flow control for specific JMS queues and topics on the broker by setting the producerFlowControl flag to false on the appropriate destination policy in the Broker configuration - e.g.

如果你喜歡,你可以通過在代理的配置中,將適當的目的地(destination)的策略(policy)中的producerFlowControl標誌設置爲false,使代理上特定的JMS隊列和主題無效,例如:

<destinationPolicy>
  <policyMap>
    <policyEntries>
      <policyEntry topic="FOO.>" producerFlowControl="false"/>
    </policyEntries>
  </policyMap>
</destinationPolicy>

see Broker Configuration.

詳情請看 代理設置 部分。

Note that, since the introduction of the new file cursor in ActiveMQ 5.x, non-persisted messages are shunted into the temporary file store to reduce the amount of memory used for non-persistent messaging. As a result, you may find that a queue's memoryLimit is never reached, as the cursor doesn't use very much memory. If you really do want to keep all your non-persistent messages in memory, and stop producers when the limit is reached, you should configure the <vmQueueCursor>.

注意,自從ActiveMQ 5.x中引入新的文件遊標之後,非持久化消息被分流到了臨時文件存儲中,以此來減少非持久化消息傳送使用的內存總量。結果就是,你可能會發現一個隊列的內存限制永遠達不到,因爲遊標不需要使用太多的內存。如果你真的想把所有的非持久化消息存放在內存中,並在達到內存限制的時候停掉生產者,你需要配置<vmQueueCursor>。

<policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb">    
  <pendingQueuePolicy>
    <vmQueueCursor/>
  </pendingQueuePolicy>
</policyEntry>

The fragment above will ensure that all non-persistent queue messages are kept in memory, with each queue having a limit of 1Mb.

上面的片段可以保證所有的非持久化隊列消息都保存在內存中,每一個隊列的內存限制爲1Mb。

How Producer Flow Control works 生產者流量控制是如何工作的

If you are sending a persistent message (so that a response of the OpenWire Message is expected then the broker will send the producer aProducerAck message. This informs the producer that the previous sending window has been processed, so that it can now send another window. Its kinda like consumer acks but in reverse.

如果你發送一條持久化消息(這樣就會有一個對 OpenWire 響應消息的期望),代理將會給生產者發送一個應答消息。該消息會通知生產者其先前的發送窗口已經被處理了,所以它現在可以發送另外的窗口了。這有點像消費者應答,不過是反向的。

Advantage 優勢

So a nice producer might wait for a producer ack before sending more data, to avoid flooding the broker (and forcing the broker to block the entire connection if a slow consumer occurs). To see how this works in source code, check out the ActiveMQMessageProducer code.

所以,一個友好的生產者可以再發送更多的數據之前,等待生產者應答,以此來避免對代理的衝擊(並且如果出現了一個比較慢的消費者,強制代理阻塞整個連接)。如果你想知道這部分的源代碼是怎麼實現的,可以看一下 ActiveMQMessageProducer 的代碼。

Though a client can ignore the producer ACKs altogether and the broker should just stall the transport if it has to for slow consumer handling; though this does mean it'll stall the entire connection.

然而一個客戶端可以完全忽略生產者的應答消息,並且處理慢消費者的時候,代理可以在需要的時候拖延傳送;雖然這意味着它將拖延整個連接。

Configure Client-Side Exceptions 配置客戶端的異常

An alternative to the indefinite blocking of the send() operation when no space is free on the broker is to instead configure that an exception to be thrown on the client-side. By configuring the sendFailIfNoSpace property to true, the broker will cause the send() operation to fail with ajavax.jms.ResourceAllocationException, which will propagate to the client. Below is an example of this configuration:

應對代理空間不足,而導致不確定的阻塞 send()操作的一種替代方案,就是將其配置成客戶端拋出的一個異常。通過將sendFailIfNoSpace屬性設置爲true,代理將會引起send()方法失敗,並拋出javax.jms.ResourceAllocationException異常,傳播到客戶端。下面是一個配置的示例:

  1. <systemUsage>
  2. <systemUsage sendFailIfNoSpace="true">
  3. <memoryUsage>
  4. <memoryUsage limit="20 mb"/>
  5. </memoryUsage>
  6. </systemUsage>
  7. </systemUsage>

The advantage of this property is that the client can catch the javax.jms.ResourceAllocationException, wait a bit and retry the send() operation instead of just hanging indefinitely.

這個屬性的好處是,客戶端可以捕獲javax.jms.ResourceAllocationException異常,稍等一下,並重試send()操作,而不是無限期地傻等下去。

Starting in version 5.3.1 the sendFailIfNoSpaceAfterTimeout property has been added. This property causes the send() operation to fail with an exception on the client-side, but only after waiting the given amount of time. If space on the broker is still not freed after the configured amount of time, only then does the send() operation fail with an exception to the client-side. Below is an example:

 從5.3.1版本之後,sendFailIfNoSpaceAfterTimeout 屬性被加了進來。這個屬性同樣導致send()方法失敗,並在客戶端拋出異常,但僅當等待了指定時間之後才觸發。如果在配置的等待時間過去之後,代理上的空間仍然沒有被釋放,僅當這個時候send()方法纔會失敗,並且在客戶端拋出異常。下面是一個示例:

  1. <systemUsage>
  2. <systemUsage sendFailIfNoSpaceAfterTimeout="3000">
  3. <memoryUsage>
  4. <memoryUsage limit="20 mb"/>
  5. </memoryUsage>
  6. </systemUsage>
  7. </systemUsage>

The timeout is defined in milliseconds so the example above waits for three seconds before failing the send() operation with an exception to the client-side. The advantage of this property is that it will block for the configured amount of time instead of failing immediately or blocking indefinitely. This property offers not only an improvement on the broker-side, but also an improvement for the client so it can catch the exception, wait a bit and retry the send() operation.

定義超時的單位是毫秒,所以上面的例子將會在使send()方法失敗並對客戶端拋出異常之前,等待三秒。這個屬性的優點是,它僅僅阻塞配置指定的時間,而不是立即另發送失敗,或者無限期阻塞。這個屬性不僅在代理端提供了一個改進,還對客戶端提供了一個改進,使得客戶端能捕獲異常,等待一下並重試send()操作。

Disabling Flow Control 使流量控制無效

A common requirement is to disable flow control so that message dispatching continues until all available disk is used up by pending messages (whether persistent or non persistent messaging is configured). To do this enable Message Cursors.

一個普遍的需求是使流量控制無效,使得消息分發能夠持續,直到所有可用的磁盤被掛起(pending)的消息耗盡(無論是持久化的還是配置了非持久化的)。要這樣做,你可以使用消息遊標(Message Cursors)。

System usage 系統佔用

You can also slow down producers via some attributes on the <systemUsage> element. Take a look at the following example:

你還可以通過<systemUsage>元素的一些屬性來減慢生產者。來看一眼下面的例子:

  1. <systemUsage>
  2. <systemUsage>
  3. <memoryUsage>
  4. <memoryUsage limit="64 mb" />
  5. </memoryUsage>
  6. <storeUsage>
  7. <storeUsage limit="100 gb" />
  8. </storeUsage>
  9. <tempUsage>
  10. <tempUsage limit="10 gb" />
  11. </tempUsage>
  12. </systemUsage>
  13. </systemUsage>

You can set limits of memory for NON_PERSISTENT messages, disk storage for PERSITENT messages and total usage for temporary messages, the broker will use before it slowdown producers. Using the default settings shown above, the broker will block the send() call until some messages are consumed and space becomes available on the broker. The default values are shown above, you will probably need to increase these values for your environment.

你可以爲非持久化的消息(NON_PERSISTENT messages)設置內存限制,爲持久化消息(PERSISTENT messages)設置磁盤空間,以及爲臨時消息設置總的空間,代理將在減慢生產者之前使用這些空間。使用上述的默認設置,代理將會一直阻塞sen()方法的調用,直至一些消息被消費,並且代理有了可用空間。默認值如上例所述,你可能需要根據你的環境增加這些值。

原文地址:http://blog.csdn.net/neareast/article/details/7581855

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章