高可用進程內緩存設計和實現【二】

在系統設計中，爲了提升性能往往需要使用到緩存，分佈式緩存效率已經很高了，例如常用的redis以及memcache，但是對於極高併發，對響應要求極高的系統，則需要使用進程內緩存，下面將進程內緩存和分佈式緩存進行了對比：

優點

缺點

方案

應用場景

進程間緩存

1.適合高併發場景，非高併發不建議使用。

2.無網絡開銷。

1.難以保證數據一致性。

2.緩存數量量大的情況下，頻繁的淘汰數據，GC可能會影響系統性能。

1.ConcurrentHashMap, ThreadLocal, guava cache.

2.http/rpc通知，一旦通知失敗導致數據不一致。

3.MQ，引入中間件，複雜。

4.進程主動拉取，但是要考慮到版本控制，減少不必要的拉取。

1.純只讀數據（配置數據）或者數據量小且頻率可預見的訪問場景。

2.高併發。

3.允許一定時間的數據不一致。

分佈式緩存

數據一致性

網絡和序列化開銷。

redis memcache

1.數據強一致性

在我們的系統中有這麼一個場景，即有一個白名單要在系統入口進行查詢，但是白名單的量很細小，只佔用戶數的1%%，且一個用戶的白名單有多個類別，需要查詢多次。每次進來都需要過白名單，因此，將白名單放在了分佈式緩存中，但是數量小，查詢次數多，變化少，這種場景符合進程間緩存的特點，提升高併發系統的響應性能，當然進程間緩存也適用於將配置信息拉取到內存中進行計算和過濾的場景，例如風控系統，就是將配置和策略定時拉取到進程內來進行計算的，下面我們通過主動拉取加上版本控制的方式來進行實現。

首先看一下抽象代碼類AbstractLocalCache.java：

package com.csdn.net.local.cache;

import com.csdn.net.local.cache.redis.RedisService;
import org.apache.commons.lang3.concurrent.BasicThreadFactory;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.annotation.Autowired;

import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.TimeUnit;


public abstract class AbstractLocalCache<T> implements InitializingBean {
    private static final ILog logger = LogFactory.getLog("LOCAL-CACHE");

    private static ScheduledExecutorService executorService = new ScheduledThreadPoolExecutor(1,
            new BasicThreadFactory.Builder().namingPattern("example-schedule-pool-%d").daemon(true).build());

    /**
     * 緩存更新版本cacheService
     */
    @Autowired
    protected RedisService versionRedisCache;

    /**
     * codis版本的key
     */
    protected String versionKey;

    /**
     * 更新緩存任務執行頻率
     * TimeUnit.SECONDS
     */
    protected long latency = 10 * 60;

    /**
     * 爲防止version失效，增加的保底更新週期設置
     * TimeUnit.MILLISECONDS
     * 60分鐘強制更細一次
     */
    private long basicLatency = 60 * 60 * 1000;

    /**
     * 是否通過版本更新，控制數據更新加載
     */
    protected boolean versionControl = true;

    /**
     * 上次緩存更新時間戳
     */
    private volatile long lastLoadTime = 0;

    private volatile long currentVersion = 0;

    /**
     * 當有數據更新時，更新redis狀態
     *
     * @return
     */
    public void mark(boolean changed) {
        if (changed) {
            try {
                versionRedisCache.incr(versionKey, 1);
            } catch (Exception e) {
                logger.error("AbstractLocalCache mark error, versionKey=" + versionKey);
            }
        }
    }

    public String getVersion() {
        return String.valueOf(versionRedisCache.get(versionKey));
    }

    /**
     * 判斷狀態是否發生了變化
     * version控制 + 保底定時更新
     *
     * @return
     */
    private boolean isLoad() {
        try {
            Object value = versionRedisCache.get(versionKey);
            logger.info("start execute isLoad, version in redis: " + value + ", currentVersion=" + currentVersion);
            if (null == value) {
                currentVersion = versionRedisCache.incr(versionKey, 1);
            } else {
                long valueOnRedis = Long.valueOf(String.valueOf(value));

                if (currentVersion == valueOnRedis) {
                    // 防止version失效，強制定時更新
                    long currentTime = System.currentTimeMillis();

                    logger.info(String.format("executing isLoad, currentTime=%s||lastLoadTime=%s||basicLatency=%s", currentTime,
                            lastLoadTime, basicLatency));

                    if ((currentTime - lastLoadTime) > basicLatency) {
                        return true;
                    }

                    return false;
                }

                currentVersion = valueOnRedis;
                return true;
            }
        } catch (Exception e) {
            logger.error("AbstractLocalCache mark exception , errmsg=" + e.getMessage() + ", versionKey=" + versionKey);
        }

        return true;
    }

    private class LoadDataTask implements Runnable {

        @Override
        public void run() {
            try {

                if (versionControl) {
                    if (isLoad()) {
                        lastLoadTime = System.currentTimeMillis();
                        doLoad();
                    }
                } else {
                    doLoad();
                }
            } catch (Exception e) {
               e.printStackTrace();
            }
        }

    }

    /**
     * 執行數據加載任務
     */
    protected abstract void doLoad() throws Exception;

    /**
     * 初始化相關配置
     */
    public abstract void init();

    @Override
    public void afterPropertiesSet() throws Exception {
        init();
        doLoad();
        executorService.scheduleAtFixedRate(new LoadDataTask(), 30, latency, TimeUnit.SECONDS);
    }
}

解析：

定時輪詢本地cache：初始化，執行load操作，然後使用JUC裏面的定時任務線程來定時加載任務。一般檢查數據是否更新的方式是，定時（例如每分鐘）load數據，更新本地cache，這種方案有個明顯的缺點在於，不管數據有沒有更新，定時都會load到本地來，這就需要一個能感知數據變化的狀態，定時去檢查這個狀態，只有和本地的狀態不一致時，纔去load。當然也有一段時間強制更新的機制。
優化：本實現中使用Redis來保存該狀態，當然也可以用zk/mysql來保存。下面看一下緩存實現類WhitelistCache.java：

package com.csdn.net.local.cache;

import com.csdn.net.local.cache.constant.Common;
import com.csdn.net.local.cache.controller.CacheController;
import com.csdn.net.local.cache.entity.RiskAssetAuditWhitelist;
import com.csdn.net.local.cache.service.WhitelistService;
import com.csdn.net.local.cache.model.WhiteListVO;
import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.stereotype.Service;

import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;


/**
 * Whitelist: 數據庫實體類
 */
@Service("whitelistCache")
public class WhitelistCache extends AbstractLocalCache<Whitelist> {

    private static final ILog logger = LogFactory.getLog("LOCAL-CACHE");

    public final static String VERSION_KEY = Common.AUDIT_WHITE_LIST_VERSION;

    @Autowired
    private WhitelistService whitelistService;


    private Map<String, String> cacheMap = new ConcurrentHashMap<>();

    @Override
    public void init() {
        super.versionKey = VERSION_KEY;
        super.latency = 15;
    }

    @Override
    protected void doLoad() throws Exception {

        List<Whitelist> list = whitelistService.queryAll();

        logger.info("Whitelist from db = " + list);

        for (Whitelist whitelist : list) {

            //白名單業務邏輯，可以忽略
            String bizId = whitelist.getCardId().trim().toUpperCase();
            String cacheKey = Common.WHITE_LIST_PREFIX + whitelist.getMtype() + "_" + Common.getMD5("1_" + bizId);
            String status = whitelist.getStatus();
            if ("1".equals(status)) {
                cacheMap.put(cacheKey, whitelist.getAuditRule());
            } else {
                cacheMap.remove(cacheKey);
            }
        }
        logger.info("cacheMap = " + cacheMap);
    }

    public String getCacheByKey(String key) {

        return cacheMap.get(key);
    }

    public Map<String, String> getCacheAll() {

        return cacheMap;
    }
}

WhiteCache進程自AbstractLocalCache，裏面通過ConcurrentHashMap來保存從數據庫中加載的數據。至於爲什麼用ConcurrentHashMap當然是保證其緩存線程安全性，下面看一下Controller：

package com.csdn.net.local.cache.controller;

import com.alibaba.fastjson.JSONObject;
import com.csdn.net.local.cache.constant.Common;
import com.csdn.net.local.cache.WhitelistCache;
import com.csdn.net.local.cache.model.WhiteListVO;
import com.csdn.net.local.cache.service.WhitelistService;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.lang.reflect.Field;

import static org.springframework.web.bind.annotation.RequestMethod.GET;
import static org.springframework.web.bind.annotation.RequestMethod.POST;

@Controller
@RequestMapping("/sec/fast/localcache")
public class CacheController extends CommonController {

    @Autowired
    private WhitelistService whitelistService;

    @Autowired
    private WhitelistCache whitelistCache;

    @ResponseBody
    @RequestMapping(value = "/whitelist/add", method = POST)
    public String add(HttpServletRequest request,
                      HttpServletResponse response,
                      @RequestBody String body) {
        WhiteListVO whiteListVO = JSONObject.parseObject(body, WhiteListVO.class);
        try {
            if (whiteListVO.getDriverId() != null) {
                whiteListVO = (WhiteListVO) isAllFieldNull(whiteListVO);
                return renderJsonSuccess(response, whitelistService.save(whiteListVO));
            }
            return renderJson(response, Common.PARAM_ERROR);
        } catch (Exception e) {
            return renderJson(response, Common.INTERNAL_ERROR);
        }
    }

    @ResponseBody
    @RequestMapping(value = "/whitelist/del", method = POST)
    public String delete(HttpServletRequest request,
                         HttpServletResponse response,
                         @RequestParam(value = "id") String id) {

        if (null != id) {
            return renderJsonSuccess(response, whitelistService.delete(id));
        }

        return renderJson(response, Common.INTERNAL_ERROR);
    }


    @ResponseBody
    @RequestMapping(value = "/whitelist/all", method = GET)
    public String queryAll(HttpServletResponse response) {

        return renderJsonSuccess(response, whitelistCache.getCacheAll());
    }

    @ResponseBody
    @RequestMapping(value = "/whitelist/version", method = GET)
    public String version(HttpServletResponse response) {

        return renderJsonSuccess(response, Common.AUDIT_WHITE_LIST_VERSION + ":"+ whitelistCache.getVersion());
    }

    public static Object isAllFieldNull(Object obj) throws Exception {
        // 取到obj的class, 並取到所有屬性

        Field[] fs = obj.getClass().getDeclaredFields();

        // 遍歷所有屬性
        for (Field f : fs) {
            f.setAccessible(true);
            if (f.get(obj) == null) {
                f.set(obj, "");
            }
        }

        return obj;
    }
}

最後看一下WhiteService和WhiteServiceImpl:

package com.csdn.net.local.cache.service;

import com.csdn.net.local.cache.entity.RiskAssetAuditWhitelist;
import com.csdn.net.local.cache.model.WhiteListVO;

import java.util.List;


public interface WhitelistService {

    List<RiskAssetAuditWhitelist> queryAll();

    int save(WhiteListVO whiteListVO);

    int delete(String id);

    int update(WhiteListVO whiteListVO);


}

package com.csdn.net.local.cache.service.impl;

import com.csdn.net.local.cache.WhitelistCache;
import com.csdn.net.local.cache.dao.RiskAssetAuditWhitelistMapper;
import com.csdn.net.local.cache.entity.RiskAssetAuditWhitelist;
import com.csdn.net.local.cache.service.WhitelistService;
import com.csdn.net.local.cache.model.WhiteListVO;

import net.sf.cglib.beans.BeanCopier;

import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;

import javax.annotation.Resource;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;

@Service
public class WhitelistServiceImpl implements WhitelistService {

    @Resource
    private RiskAssetAuditWhitelistMapper whiteListMapper;

    @Resource
    private WhitelistCache whitelistCache;

    @Override
    public List<RiskAssetAuditWhitelist> queryAll() {
        return whiteListMapper.queryAll();
    }

    @Override
    @Transactional(propagation = Propagation.REQUIRED, readOnly = false, rollbackFor = Exception.class)
    public int save(WhiteListVO whiteListVO) {
        RiskAssetAuditWhitelist riskAssetSillAuditWhitelist = new RiskAssetAuditWhitelist();

        BeanCopier.create(whiteListVO.getClass(), riskAssetSillAuditWhitelist.getClass(), false).copy(whiteListVO,riskAssetSillAuditWhitelist,null);

        String date = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());
        riskAssetSillAuditWhitelist.setCreateTime(date);
        riskAssetSillAuditWhitelist.setUpdateTime(date);
        int ret = whiteListMapper.insertSelective(riskAssetSillAuditWhitelist);

        whitelistCache.mark(true);

        return ret;
    }

    @Override
    @Transactional(propagation = Propagation.REQUIRED, readOnly = false, rollbackFor = Exception.class)
    public int delete(String id) {

        RiskAssetAuditWhitelist whitelist = whiteListMapper.selectByPrimaryKey(Long.parseLong(id));

        whitelist.setStatus("0");

        int ret = whiteListMapper.updateByPrimaryKey(whitelist);

        whitelistCache.mark(true);

        return ret;
    }

    @Override
    @Transactional(propagation = Propagation.REQUIRED, readOnly = false, rollbackFor = Exception.class)
    public int update(WhiteListVO whiteListVO) {
        RiskAssetAuditWhitelist riskAssetSillAuditWhitelist = new RiskAssetAuditWhitelist();
        BeanCopier.create(whiteListVO.getClass(), riskAssetSillAuditWhitelist.getClass(), false).copy(whiteListVO,riskAssetSillAuditWhitelist,null);

        riskAssetSillAuditWhitelist.setUpdateTime(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date()));

        whitelistCache.mark(true);

        return whiteListMapper.updateByPrimaryKey(riskAssetSillAuditWhitelist);
    }
}

最後看一下結果：

首先啓動服務，拉取數據庫中的六條記錄，調用接口"/sec/fast/localcache/whitelist/all"：

結果：

{
    "result": {
        "wl_1_50cd243d6bc9ca7a90c39c3fa84aaaeb": "2",
        "wl_3_d3226abd9388aead8e972816be481a1e": "-2",
        "wl_1_a2804b7864f0fdbff0a8f1298a4e2300": "2",
        "wl_1_7965fae89a38d26d63b61632119cfa75": "2",
        "wl_1_7aa7bf2bc7996ed3d772ccf6347c6a01": "2",
        "wl_1_50df7489e1d741f80c9089821e3b0424": "-2"
    },
    "errno": "0",
    "errmsg": "ok"
}

日誌：

[INFO][2019-06-21T23:00:50.789+0800][com.csdn.net.local.cache.AbstractLocalCache.isLoad(AbstractLocalCache.java:92)] start execute isLoad, version in redis: 4, currentVersion=4
[INFO][2019-06-21T23:00:50.792+0800][com.csdn.net.local.cache.AbstractLocalCache.isLoad(AbstractLocalCache.java:102)] executing isLoad, currentTime=1561129250791||lastLoadTime=1561129235939||basicLatency=3600000

下面刪除一條記錄，調用"/sec/fast/localcache/whitelist/del"：

日誌：

[INFO][2019-06-21T23:03:20.788+0800][com.csdn.net.local.cache.AbstractLocalCache.isLoad(AbstractLocalCache.java:92)] start execute isLoad, version in redis: 5, currentVersion=4
[INFO][2019-06-21T23:03:35.786+0800][com.csdn.net.local.cache.AbstractLocalCache.isLoad(AbstractLocalCache.java:92)] start execute isLoad, version in redis: 5, currentVersion=5
[INFO][2019-06-21T23:03:35.787+0800][com.csdn.net.local.cache.AbstractLocalCache.isLoad(AbstractLocalCache.java:102)] executing isLoad, currentTime=1561129415787||lastLoadTime=1561129400788||basicLatency=3600000

再次調用接口"/sec/fast/localcache/whitelist/all"：

{
    "result": {
        "wl_1_50cd243d6bc9ca7a90c39c3fa84aaaeb": "2",
        "wl_3_d3226abd9388aead8e972816be481a1e": "-2",
        "wl_1_a2804b7864f0fdbff0a8f1298a4e2300": "2",
        "wl_1_7aa7bf2bc7996ed3d772ccf6347c6a01": "2",
        "wl_1_50df7489e1d741f80c9089821e3b0424": "-2"
    },
    "errno": "0",
    "errmsg": "ok"
}

Author：憶之獨秀

Email：[email protected]

註明出處：https://blog.csdn.net/lavorange/article/details/93234223

高可用進程內緩存設計和實現【二】

【Redis】Redis最大連接數maxTotal設置過小的問題

【Mysql】記一次limit優化

【Mysql】Explain關鍵字分析SQL性能

【JVM】OOMError之Unable to create new native thread

【微服務穩定性之限流降級】【二】RateLimiter實戰

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結