tornado之多進程服務

tornado多進程開啓

在前文"tornado之hello world"中簡單介紹過tornado的基本使用。tornado默認啓動的是單進程，啓動方式如下：

if __name__ == "__main__":
    application = tornado.web.Application([(r"/", MainHandler)])
    # application.listen(8888)
    server = HTTPServer(application)
    server.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

如果要開啓多進程，可進行如下修改：

if __name__ == "__main__":
    application = tornado.web.Application([(r"/", MainHandler)])
    server = HTTPServer(application)
    server.bind(8888)
    server.start(2)
    tornado.ioloop.IOLoop.instance().start()

啓動後生成了3個進程(注意關閉debug模式)。start函數中參數說明如下：

If num_processes is ``None`` or <= 0, we detect the number of cores
available on this machine and fork that number of child
processes. If num_processes is given and > 1, we fork that
specific number of sub-processes.

意思是，當參數小於等於０時，則根據當前機器的cpu核數來創建子進程，大於１時直接根據指定參數創建子進程。
看一下bind函數的說明：

def bind(self, port, address=None, family=socket.AF_UNSPEC, backlog=128,
         reuse_port=False):
    """Binds this server to the given port on the given address.

    To start the server, call `start`. If you want to run this server
    in a single process, you can call `listen` as a shortcut to the
    sequence of `bind` and `start` calls.

    Address may be either an IP address or hostname.  If it's a hostname,
    the server will listen on all IP addresses associated with the
    name.  Address may be an empty string or None to listen on all
    available interfaces.  Family may be set to either `socket.AF_INET`
    or `socket.AF_INET6` to restrict to IPv4 or IPv6 addresses, otherwise
    both will be used if available.

    The ``backlog`` argument has the same meaning as for
    `socket.listen <socket.socket.listen>`. The ``reuse_port`` argument
    has the same meaning as for `.bind_sockets`.

    This method may be called multiple times prior to `start` to listen
    on multiple ports or interfaces.

    .. versionchanged:: 4.4
       Added the ``reuse_port`` argument.
    """

也就是說，當start函數的參數爲１時，listen函數可以代替bind和start。

多進程開啓原理

對啓動代碼進行分析：
先創建一個HTTPServer實例。然後在bind函數中，調用了bind_sockets方法來創建sockets。接着在start方法中，調用process.fork_processes(num_processes)方法來創建子進程，並將sockets通過add_sockets方法添加到server實例中。
也就是說，這一段代碼的實質如下：

if __name__ == "__main__":
    application = tornado.web.Application([(r"/", MainHandler)])
    sockets = tornado.netutil.bind_sockets(8888)
    tornado.process.fork_processes(2)
    server = HTTPServer(application)
    server.add_sockets(sockets)
    tornado.ioloop.IOLoop.instance().start()

繼續分析fork_process函數，其主要過程如下：

def start_child(i):
    pid = os.fork()
    if pid == 0:
        # child process
        _reseed_random()
        global _task_id
        _task_id = i
        return i
    else:
        children[pid] = i
        return None

for i in range(num_processes):
    id = start_child(i)
    if id is not None:
        return id
num_restarts = 0
while children:
    try:
        pid, status = os.wait()
    except OSError as e:
        if errno_from_exception(e) == errno.EINTR:
            continue
        raise
    if pid not in children:
        continue
    id = children.pop(pid)
    if os.WIFSIGNALED(status):
        gen_log.warning("child %d (pid %d) killed by signal %d, restarting",
                        id, pid, os.WTERMSIG(status))
    elif os.WEXITSTATUS(status) != 0:
        gen_log.warning("child %d (pid %d) exited with status %d, restarting",
                        id, pid, os.WEXITSTATUS(status))
    else:
        gen_log.info("child %d (pid %d) exited normally", id, pid)
        continue
    num_restarts += 1
    if num_restarts > max_restarts:
        raise RuntimeError("Too many child restarts, giving up")
    new_id = start_child(id)
    if new_id is not None:
        return new_id
# All child processes exited cleanly, so exit the master process
# instead of just returning to right after the call to
# fork_processes (which will probably just start up another IOLoop
# unless the caller checks the return value).
sys.exit(0)

從這個源碼可以看出，子進程的開啓實際上是調用的linux系統提供的fork功能。當子進程存在時，父進程通過一個while循環來處理子進程的終止信號，嘗試重啓已經停止的子進程，直到超過默認的最大重啓次數100。