前一段時間2014北京PyCon大會吐槽頗多,所以我就到InfoQ上找了找2013的大會視頻,對網絡射擊手遊High Noon 2基於Python的服務器架構的視頻挺感興趣,尤其是遊戲服務器中的0 downtime,原理他們底層不是原生的socket,
而是基於ZeroMq的socket,由於ZeroMq的短線自動重連可以滿足遊戲服務器的熱啓動,不需要代碼層面的熱啓動,熱更新,當更新代碼完成後直接重啓服務器,之前未處理的請求會繼續處理。瞬間覺得非常高大上,於是最近一段時間回家一直研究ZeroMq,在guide的LRU這邊停留了較長時間,本篇就是談談ZeroMq LRU算法中間件。
對普通的請求回覆代理,也就是使用了ROUTER-DEALER模式比較好理解,但這種方式有個天生缺點,DEALER會使用負載均衡的方式將客戶端的請求轉發給服務器端,由於服務器的處理能力各不相同,就會導致有些服務器很忙,有些很閒,這不是我們想看到的,我們希望能壓榨所有服務器能力,所以就出現了通用型的LRU算法。
這裏使用ROUTER-ROUTER模式,剛開始你可能很詫異,且聽我慢慢道來。
那如何能充分使用每臺服務器性能呢?
1.當woker啓動時告訴backend自己已準備好,將該worker加入work_queue可用隊列中
2.當client請求時,從work_queue中取出一個work,把請求交給該work處理
3.當work處理好後把響應發給client,並把自己再次加入到work_queue中
也就是使用work隊列,記錄可用work,只要work完成請求就代表已空閒,再次加入work隊列。
那爲什麼backend需要使用ROUTER模式呢?關鍵點是當我要把client來的請求發給固定的work,而只有ROUTER模式具有路由標識功能,明白了這點,代碼也就容易了。
"""
Least-recently used (LRU) queue device
Clients and workers are shown here in-process
Author: Guillaume Aubert (gaubert) <guillaume(dot)aubert(at)gmail(dot)com>
"""
from __future__ import print_function
import threading
import time
import zmq
NBR_CLIENTS = 10
NBR_WORKERS = 3
def worker_thread(worker_url, context, i):
""" Worker using REQ socket to do LRU routing
work的響應消息格式爲
--------------------
|Frame0| Client-idntity
--------------------
|Frame1|
--------------------
|Frame2| OK
"""
socket = context.socket(zmq.REQ)
# Set the worker identity
socket.identity = (u"Worker-%d" % (i)).encode('ascii')
socket.connect(worker_url)
# Tell the borker we are ready for work
socket.send(b"READY")
try:
while True:
address = socket.recv()
empty = socket.recv()
request = socket.recv()
print("%s: %s\n" % (socket.identity.decode('ascii'), request.decode('ascii')), end='')
socket.send(address, zmq.SNDMORE)
socket.send(b"", zmq.SNDMORE)
socket.send(b"OK")
except zmq.ContextTerminated:
# context terminated so quit silently
return
def client_thread(client_url, context, i):
""" Basic request-reply client using REQ socket
client的消息格式爲
--------------------
|Frame0| Client-1
--------------------
|Frame1|
--------------------
|Frame2| HELLO
"""
socket = context.socket(zmq.REQ)
socket.identity = (u"Client-%d" % (i)).encode('ascii')
socket.connect(client_url)
# Send request, get reply
socket.send(b"HELLO")
reply = socket.recv()
print("%s: %s\n" % (socket.identity.decode('ascii'), reply.decode('ascii')), end='')
def main():
""" main method """
url_worker = "inproc://workers"
url_client = "inproc://clients"
client_nbr = NBR_CLIENTS
# Prepare our context and sockets
context = zmq.Context()
frontend = context.socket(zmq.ROUTER)
frontend.bind(url_client)
backend = context.socket(zmq.ROUTER)
backend.bind(url_worker)
# create workers and clients threads
for i in range(NBR_WORKERS):
thread = threading.Thread(target=worker_thread, args=(url_worker, context, i, ))
thread.start()
for i in range(NBR_CLIENTS):
thread_c = threading.Thread(target=client_thread, args=(url_client, context, i, ))
thread_c.start()
# Logic of LRU loop
# - Poll backend always, frontend only if 1+ worker ready
# - If worker replies, queue worker as ready and forward reply
# to client if necessary
# - If client requests, pop next worker and send request to it
# Queue of available workers
available_workers = 0
workers_list = []
# init poller
poller = zmq.Poller()
# Always poll for worker activity on backend
poller.register(backend, zmq.POLLIN)
# Poll front-end only if we have available workers
poller.register(frontend, zmq.POLLIN)
while True:
socks = dict(poller.poll())
# Handle worker activity on backend
if (backend in socks and socks[backend] == zmq.POLLIN):
# Queue worker address for LRU routing
worker_addr = backend.recv()
assert available_workers < NBR_WORKERS
# add worker back to the list of workers
available_workers += 1
workers_list.append(worker_addr)
# Second frame is empty
empty = backend.recv()
assert empty == b""
# Third frame is READY or else a client reply address
client_addr = backend.recv()
# If client reply, send rest back to frontend
if client_addr != b"READY":
# Following frame is empty
empty = backend.recv()
assert empty == b""
reply = backend.recv()
frontend.send(client_addr, zmq.SNDMORE)
frontend.send(b"", zmq.SNDMORE)
frontend.send(reply)
client_nbr -= 1
if client_nbr == 0:
break # Exit after N messages
# poll on frontend only if workers are available
if available_workers > 0:
if (frontend in socks and socks[frontend] == zmq.POLLIN):
# Now get next client request, route to LRU worker
# Client request is [address][empty][request]
client_addr = frontend.recv()
empty = frontend.recv()
assert empty == b""
request = frontend.recv()
# Dequeue and drop the next worker address
available_workers -= 1
worker_id = workers_list.pop()
"""worker_id就是work的標識,也就是需要發給worker_id,所以backend需要使用ROUTER模式
在所有消息之間zmq需要一個空消息作爲標識,當work接受到請求時會直接讀到第一個空消息,
也就是work接受的第一個消息就是client_addr,然後work再把處理的響應發給client_addr,
當backend收到消息後,直接通過forentend轉發給client_addr,這也是forentend也需要是ROUTER
模式的原因
"""
backend.send(worker_id, zmq.SNDMORE)
backend.send(b"", zmq.SNDMORE)
backend.send(client_addr, zmq.SNDMORE)
backend.send(b"", zmq.SNDMORE)
backend.send(request)
# Out of infinite loop: do some housekeeping
frontend.close()
backend.close()
context.term()
if __name__ == "__main__":
main()
"""
當需要具體轉發的時候,就是ROUTER大顯身手的時候
"""