通過學習openstack源碼,發現os代碼裏服務線程大都是green thread,這是個什麼東西?我們先看看os裏怎麼用的:
以nova-api服務爲例,這是一個提供rest api接口的服務,也就是爲用戶系統所使用,這個模塊在os代碼裏由wsgi.py提供,下邊是wsgi模塊的Server定義:
class Server(object):
"""Server class to manage multiple WSGI sockets and applications."""
def __init__(self, threads=1000):
logging.basicConfig()
self.pool = eventlet.GreenPool(threads)
def start(self, application, port, host='0.0.0.0', backlog=128):
"""Run a WSGI server with the given application."""
arg0 = sys.argv[0]
logging.audit(_("Starting %(arg0)s on %(host)s:%(port)s") % locals())
socket = eventlet.listen((host, port), backlog=backlog)
self.pool.spawn_n(self._run, application, socket)
def wait(self):
"""Wait until all servers have completed running."""
try:
self.pool.waitall()
except KeyboardInterrupt:
pass
def _run(self, application, socket):
"""Start a WSGI server in a new green thread."""
logger = logging.getLogger('eventlet.wsgi.server')
eventlet.wsgi.server(socket, application, custom_pool=self.pool,
log=WritableLogger(logger))
如類的介紹,這是一個管理sockets和應用程序的類,那這個類是如何實現的呢?我們先看看該類的構造函數,類實例化時有一個threads參數,傳給eventlet.GreenPool,這是幹什麼呢?eventlet又是什麼?
eventlet
官方的解釋:
Eventlet is a concurrent networking library for Python that allows you to change how you run your code, not how you write it.
It uses epoll or libevent for highly scalable non-blocking I/O. Coroutines ensure that the developer uses a blocking style of programming that is similar to threading, but provide the benefits of non-blocking I/O. The event dispatch is implicit, which means you can easily use Eventlet from the Python interpreter, or as a small part of a larger application.
It’s easy to get started using Eventlet, and easy to convert existing applications to use it.
關鍵幾點:一個協程庫,依賴epoll或者libevent,實現高可擴展的非阻塞網絡io程序。
安裝方式:pip install eventlet
我們來看幾個例子:
>>> import eventlet
>>> from eventlet.green import urllib2
>>> gt = eventlet.spawn(urllib2.urlopen, 'https://www.baidu.com')
>>> gt.wait()
<addinfourl at 27997320 whose fp = <socket._fileobject object at 0x1a597d0>>
>>> a=gt.wait()
>>> dir(a)
['__doc__', '__init__', '__iter__', '__module__', '__repr__', 'close', 'code', 'fileno', 'fp', 'getcode', 'geturl', 'headers', 'info', 'msg', 'next', 'read', 'readline', 'readlines', 'url']
>>> a.rul
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: addinfourl instance has no attribute 'rul'
>>> a.url
'https://www.baidu.com'
>>> a.headers
<httplib.HTTPMessage instance at 0x1aadb00>
>>> a.msg
'OK'
>>> a.read()
'<html>\r\n<head>\r\n\t<script>\r\n\t\tlocation.replace(location.href.replace("https://","http://"));\r\n\t</script>\r\n</head>\r\n<body>\r\n\t<noscript><meta http-equiv="refresh" content="0;url=http://www.baidu.com/"></noscript>\r\n</body>\r\n</html>'
>>>
>>>
再看:
#!/usr/bin/env python
"""
This is a simple web "crawler" that fetches a bunch of urls using a pool to
control the number of outbound connections.
"""
import eventlet
from eventlet.green import urllib2
urls = [
"https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg",
"http://python.org/images/python-logo.gif",
]
def fetch(url):
print("opening", url)
body = urllib2.urlopen(url).read()
print("done with", url)
return url, body
pool = eventlet.GreenPool(200)
for url, body in pool.imap(fetch, urls):
print("got body from", url, "of length", len(body))
運行輸出:
('opening', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg')
('opening', 'http://python.org/images/python-logo.gif')
('done with', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg')
('got body from', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg', 'of length', 10452)
('done with', 'http://python.org/images/python-logo.gif')
('got body from', 'http://python.org/images/python-logo.gif', 'of length', 2549)
[root@localhost python]#
發現是順序執行的,那如何啓動一個wsgi server呢?
官方給的是這樣的:
"""This is a simple example of running a wsgi application with eventlet.
For a more fully-featured server which supports multiple processes,
multiple threads, and graceful code reloading, see:
http://pypi.python.org/pypi/Spawning/
"""
import eventlet
from eventlet import wsgi
def hello_world(env, start_response):
if env['PATH_INFO'] != '/':
start_response('404 Not Found', [('Content-Type', 'text/plain')])
return ['Not Found\r\n']
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['Hello, World!\r\n']
wsgi.server(eventlet.listen(('', 8090)), hello_world)
運行:
[root@localhost python]# python eventlet-1.py
(17415) wsgi starting up on http://0.0.0.0:8090
(17415) accepted ('172.30.59.209', 57684)
172.30.59.209 - - [19/Apr/2020 10:59:10] "GET / HTTP/1.1" 200 140 0.000132
172.30.59.209 - - [19/Apr/2020 10:59:11] "GET / HTTP/1.1" 200 140 0.000099
172.30.59.209 - - [19/Apr/2020 10:59:12] "GET / HTTP/1.1" 200 140 0.000105
^Cwsgi exiting
(17415) wsgi exited, is_accepting=True
這是一個簡單的例子,我們看到基本的用法就是這樣,server函數需要一個socket參數,一個app參數,這個app需要有兩個參數,一個是env環境參數,一個是用來處理http響應的函數。,默認監聽所有地址。