聊聊storagetapper的cache

序

本文主要研究一下storagetapper的cache

cache

storagetapper/pipe/cache.go

type cacheEntry struct {
	pipe Pipe
	cfg  config.PipeConfig
}

var cache map[string]cacheEntry
var lock sync.Mutex

cache是一個cacheEntry的map，cacheEntry定義了Pipe和config.PipeConfig

CacheGet

storagetapper/pipe/cache.go

// CacheGet returns an instance of pipe with specified config from cache or
// creates new one if it's not in the cache yet
func CacheGet(pipeType string, cfg *config.PipeConfig, db *sql.DB) (Pipe, error) {
	lock.Lock()
	defer lock.Unlock()

	if cache == nil {
		cache = make(map[string]cacheEntry)
	}

	b, err := json.Marshal(cfg)
	if err != nil {
		return nil, err
	}

	h := sha256.New()
	_, _ = h.Write([]byte(pipeType + "$$$" + fmt.Sprintf("%p", db) + "$$$"))
	_, _ = h.Write(b)
	hs := fmt.Sprintf("%0x", h.Sum(nil))

	p, ok := cache[hs]
	if ok && reflect.DeepEqual(cfg, &p.cfg) {
		return p.pipe, nil
	}

	//FIXME: Implement proper collisions handling

	np, err := Create(pipeType, cfg, db)
	if err != nil {
		return nil, err
	}

	cache[hs] = cacheEntry{np, *cfg}

	log.Debugf("Created and cached new '%v' pipe (hash %v) with config: %+v. Cache size %v", pipeType, hs, *cfg, len(cache))

	return np, nil
}

CacheGet方法加鎖操作cache，首先通過sha256來對pipeType及db來作爲cache的key，然後取出cacheEntry，若存在則判斷cfg與cacheEntry的cfg是否一樣，如果一樣則返回cacheEntry的pipe；否則通過Create創建Pipe，然後放入cache中

CacheDestroy

storagetapper/pipe/cache.go

// CacheDestroy releases all resources associated with cached pipes
func CacheDestroy() {
	lock.Lock()
	defer lock.Unlock()

	for h, p := range cache {
		log.Debugf("Closing %v pipe (hash %v) with config %+v", p.pipe.Type(), h, p.cfg)
		log.E(p.pipe.Close())
	}

	cache = nil
}

CacheDestroy方法通過加鎖遍歷cache，挨個執行pipe.Close()

小結

storagetapper的cache是一個cacheEntry的map，cacheEntry定義了Pipe和config.PipeConfig；CacheGet方法會獲取cache，獲取不到則Create；CacheDestroy則會通過加鎖遍歷cache，挨個執行pipe.Close()。

doc

• storagetapper