boto線程不安全的解決辦法

boto線程不安全：

https://www.coder.work/article/368131

解決辦法：一個進程初始化一個boto客戶端

def initializer():
    global s3_client
    config.read('config/my.cfg')
    CONS_AK = config.get('default', 'aws_access_key_id')
    CONS_SK = config.get('default', 'aws_secret_access_key')
    bucket_name = config.get('default', 'bucket_name')
    host = config.get('default', 'host')
    port = config.get('default', 'port')
    s3_client = boto.connect_s3(
              aws_access_key_id=CONS_AK,
              aws_secret_access_key=CONS_SK,
              host=str(host),
              port=int(port),
              is_secure=False,
              calling_format=boto.s3.connection.OrdinaryCallingFormat()
              )
    print('init s3_client')

 

with Pool(96, initializer, ())as p:
        p.map(gen_image, params)

參考：https://docs.python.org/zh-cn/3/library/multiprocessing.html#module-multiprocessing.pool