多線程抽象編程模型（Python版）

最近需要完成一個多線程下載的工具，對其中的多線程下載進行了一個抽象，可以對所有需要使用到多線程編程的地方統一使用這個模型來進行編寫。
主要結構：
1、基於Queue標準庫實現了一個類似線程池的工具，用戶指定提交任務線程submitter與工作線程worker數目，所有線程分別設置爲後臺運行，提供等待線程運行完成的接口。
2、所有需要完成的任務抽象成task，提供單獨的無參數調用方式，供worker線程調用；task以生成器的方式作爲參數提供，供submitter調用。
3、所有需要進行線程交互的信息放在context類中。
主要實現代碼如下:

#Submitter線程類實現，主要是`task_generator`調用
class SubmitterThread(threading.Thread):
    _DEFAULT_WAIT_TIMEOUT = 2 #seconds
    def __init__(self, queue, task_gen, timeout=2):
        super(SubmitterThread, self).__init__()
        self.queue = queue
        if not isinstance(timeout, int):
            _logger.error('Thread wait timeout value error: %s, '
                          'use default instead.' % timeout)
            self.timeout = self._DEFAULT_WAIT_TIMEOUT
        self.timeout = timeout
        self.task_generator = task_gen

    def run(self):
        while True:
            try:
                task = self.task_generator.next()
                self.queue.put(task, True, self.timeout)
            except Queue.Full:
                _logger.debug('Task queue is full. %s wait %d second%s timeout' %
                              (self.name, self.timeout, 's' if (self.timeout > 1) else ''))
                break
            except (StopIteration, ValueError) as e:
                _logger.debug('Task finished')
                break

#Worker線程實現，主要就是try塊內的func調用
class WorkerThread(threading.Thread):
    _DEFAULT_WAIT_TIMEOUT = 2 #seconds
    def __init__(self, queue, timeout=2):
        super(WorkerThread, self).__init__()
        self.queue = queue
        if not isinstance(timeout, int):
            _logger.error('Thread wait timeout value error: %s, '
                          'use default instead.' % timeout)
            self.timeout = self._DEFAULT_WAIT_TIMEOUT
        self.timeout = timeout

    def run(self):
        while True:
            try:
                func = self.queue.get(True, self.timeout)
            except Queue.Empty:
                _logger.debug('Task queue is empty. %s wait %d second%s timeout' %
                              (self.name, self.timeout, 's' if (self.timeout > 1) else ''))
                break

            if not callable(func):
                time.sleep(1)
            try:
                func()
            except Exception as e:
                _logger.error('Thread %s running occurs error: %s' %
                              (self.name, e))
                print('Thread running error: %s' % e)

class Executor(object):
    """
    The really place to execute executor
    """
    thread_list = []
    submitters = 0
    workers = 0
    queue = None
    task_generator = None
    timeout = 0
    def __init__(self, task_gen, submitters=1, workers=1 , timeout=2):
        if len(self.thread_list) != 0:
            raise RuntimeError('Executor can only instance once.')
        self.queue = Queue.Queue(maxsize=submitters * 2 + workers * 2)
        self.submitters = submitters
        self.workers = workers
        self.task_generator = task_gen
        self.timeout = timeout

    def start(self):
        for i in range(self.submitters):
            submitter = SubmitterThread(self.queue, self.task_generator, self.timeout)
            self.thread_list.append(submitter)
            submitter.setName('Submitter-%d' % i)
            submitter.setDaemon(True)
            submitter.start()
        for i in range(self.workers):
            worker = WorkerThread(self.queue, self.timeout)
            self.thread_list.append(worker)
            worker.setName('Worker-%d' % i)
            worker.setDaemon(True)
            worker.start()

    def is_alive(self):
        alive = False
        for t in self.thread_list:
            if t.isAlive():
                alive = True
                break
        return alive

    def wait_to_shutdown(self):
        _logger.debug('Start to wait to shutdown')
        for t in self.thread_list:
            t.join()
            _logger.debug('Shutdown thread : %s' % t.name)

Executor類保存了線程池，提供相應接口。有了這個抽象之後，只需要實例化Executor類的對象，然後調用start方法進行多線程任務的運行。並可以用is_alive等接口再主線程內進行其他處理。
後續再使用這個抽象進行實際多線程任務的實現。