pyhton greent hread

通過學習openstack源碼,發現os代碼裏服務線程大都是green thread,這是個什麼東西?我們先看看os裏怎麼用的:

以nova-api服務爲例,這是一個提供rest api接口的服務,也就是爲用戶系統所使用,這個模塊在os代碼裏由wsgi.py提供,下邊是wsgi模塊的Server定義:

class Server(object):
    """Server class to manage multiple WSGI sockets and applications."""

    def __init__(self, threads=1000):
        logging.basicConfig()
        self.pool = eventlet.GreenPool(threads)

    def start(self, application, port, host='0.0.0.0', backlog=128):
        """Run a WSGI server with the given application."""
        arg0 = sys.argv[0]
        logging.audit(_("Starting %(arg0)s on %(host)s:%(port)s") % locals())
        socket = eventlet.listen((host, port), backlog=backlog)
        self.pool.spawn_n(self._run, application, socket)

    def wait(self):
        """Wait until all servers have completed running."""
        try:
            self.pool.waitall()
        except KeyboardInterrupt:
            pass

    def _run(self, application, socket):
        """Start a WSGI server in a new green thread."""
        logger = logging.getLogger('eventlet.wsgi.server')
        eventlet.wsgi.server(socket, application, custom_pool=self.pool,
                             log=WritableLogger(logger))

如類的介紹,這是一個管理sockets和應用程序的類,那這個類是如何實現的呢?我們先看看該類的構造函數,類實例化時有一個threads參數,傳給eventlet.GreenPool,這是幹什麼呢?eventlet又是什麼?

eventlet

官方的解釋:

Eventlet is a concurrent networking library for Python that allows you to change how you run your code, not how you write it.

It uses epoll or libevent for highly scalable non-blocking I/O. Coroutines ensure that the developer uses a blocking style of programming that is similar to threading, but provide the benefits of non-blocking I/O. The event dispatch is implicit, which means you can easily use Eventlet from the Python interpreter, or as a small part of a larger application.

It’s easy to get started using Eventlet, and easy to convert existing applications to use it.

關鍵幾點:一個協程庫,依賴epoll或者libevent,實現高可擴展的非阻塞網絡io程序。

安裝方式:pip install eventlet

我們來看幾個例子:

>>> import eventlet
>>> from eventlet.green import urllib2
>>> gt = eventlet.spawn(urllib2.urlopen, 'https://www.baidu.com')
>>> gt.wait()
<addinfourl at 27997320 whose fp = <socket._fileobject object at 0x1a597d0>>
>>> a=gt.wait()
>>> dir(a)
['__doc__', '__init__', '__iter__', '__module__', '__repr__', 'close', 'code', 'fileno', 'fp', 'getcode', 'geturl', 'headers', 'info', 'msg', 'next', 'read', 'readline', 'readlines', 'url']
>>> a.rul
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: addinfourl instance has no attribute 'rul'
>>> a.url
'https://www.baidu.com'
>>> a.headers
<httplib.HTTPMessage instance at 0x1aadb00>
>>> a.msg
'OK'
>>> a.read()
'<html>\r\n<head>\r\n\t<script>\r\n\t\tlocation.replace(location.href.replace("https://","http://"));\r\n\t</script>\r\n</head>\r\n<body>\r\n\t<noscript><meta http-equiv="refresh" content="0;url=http://www.baidu.com/"></noscript>\r\n</body>\r\n</html>'
>>>
>>>

再看:

#!/usr/bin/env python
"""
This is a simple web "crawler" that fetches a bunch of urls using a pool to
control the number of outbound connections.
"""
import eventlet
from eventlet.green import urllib2


urls = [
    "https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg",
    "http://python.org/images/python-logo.gif",
]


def fetch(url):
    print("opening", url)
    body = urllib2.urlopen(url).read()
    print("done with", url)
    return url, body


pool = eventlet.GreenPool(200)
for url, body in pool.imap(fetch, urls):
    print("got body from", url, "of length", len(body))
運行輸出:

('opening', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg')
('opening', 'http://python.org/images/python-logo.gif')
('done with', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg')
('got body from', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg', 'of length', 10452)
('done with', 'http://python.org/images/python-logo.gif')
('got body from', 'http://python.org/images/python-logo.gif', 'of length', 2549)
[root@localhost python]#

發現是順序執行的,那如何啓動一個wsgi server呢?

官方給的是這樣的:

"""This is a simple example of running a wsgi application with eventlet.
For a more fully-featured server which supports multiple processes,
multiple threads, and graceful code reloading, see:

http://pypi.python.org/pypi/Spawning/
"""

import eventlet
from eventlet import wsgi


def hello_world(env, start_response):
    if env['PATH_INFO'] != '/':
        start_response('404 Not Found', [('Content-Type', 'text/plain')])
        return ['Not Found\r\n']
    start_response('200 OK', [('Content-Type', 'text/plain')])
    return ['Hello, World!\r\n']

wsgi.server(eventlet.listen(('', 8090)), hello_world)

運行:
[root@localhost python]# python eventlet-1.py
(17415) wsgi starting up on http://0.0.0.0:8090
(17415) accepted ('172.30.59.209', 57684)
172.30.59.209 - - [19/Apr/2020 10:59:10] "GET / HTTP/1.1" 200 140 0.000132
172.30.59.209 - - [19/Apr/2020 10:59:11] "GET / HTTP/1.1" 200 140 0.000099
172.30.59.209 - - [19/Apr/2020 10:59:12] "GET / HTTP/1.1" 200 140 0.000105
^Cwsgi exiting
(17415) wsgi exited, is_accepting=True

這是一個簡單的例子,我們看到基本的用法就是這樣,server函數需要一個socket參數,一個app參數,這個app需要有兩個參數,一個是env環境參數,一個是用來處理http響應的函數。,默認監聽所有地址。

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章