python h5py 創建無限數據集、在無限數據集上添加新的數據

# -*- coding:utf-8 -*-

import h5py
import numpy as np

file = h5py.File('myunlim_dataset.hdf5','a')

dt_unlimt = file.create_dataset('data',[10,224,224,3],maxshape=[None,224,224,3],chunks=True,
compression='gzip',compression_opts=7) # 創建一個數據集對象，先把各項參數設置好，後續就不能改了，後面直接通過索引寫入數據，數據會自動根據這些屬性設置進行壓縮存儲。

# generate example data
a = np.arange(10*224*224*3).reshape(10,224,224,3)

# write a into dataset 'data',remenmber dt_unlimt is a 'data' dataset obj.
dt_unlimt[0:10] = a # write in by slice !

# now, we need to write new data to dataset 'data', and shape is [100,224,224,3],for example:

b = np.arange(100*224*224*3).reshape(100,224,224,3)

# before we write in new data, we need to resize dataset 'data' 's total shape first:
dt_unlimt.resize((110,224,224,3)) # 這裏的resize的參數是dataset的總的shape，不僅僅是新寫入數據集的shape。
dt_unlimt[10:110] = b # write in by slice! 記住，python切片是左閉右開的

# 寫完數據不要忘記關閉！
file.close()

# That's OK !