SpringBoot整合JPA實現多數據源及讀寫分離

SpringBoot整合JPA實現多數據源及讀寫分離

項目地址：https://github.com/baojingyu/spring-boot-jpa-dynamic-datasource

本項目使用 SpringBoot 和 SpringData JPA 實現多數據源，動態數據源的切換

前段時間寫 MySQL 主從複製的技術分享，我就在想，既然主從複製都搭建好了，不寫個 Demo 玩玩讀寫分離，好像差點意思哼～ 於是就有了這麼個 Demo Project。

一、什麼是讀寫分離？

讀寫分離其實就是將數據庫分爲主從庫，簡單點說是主庫處理事務性增、改、刪操作，多個從庫處理查詢操作。主庫的事務性操作導致的變更通過BinLog 日誌同步到集羣中的從庫。

二、爲什麼要讀寫分離？

其實最主要是減輕數據庫的壓力。數據庫的寫操作比較耗時，如果沒有進行讀寫分離，寫操作將會影響到讀操作的效率。

三、什麼時候要讀寫分離？

如果程序讀多寫少，那麼可以考慮讀寫分離，利用數據庫主從同步，能減輕數據庫壓力，提高性能。

四、讀寫分離方案

數據庫讀遠大於寫，查詢多的情況，那就得考慮主庫負責寫操作，多個從庫負責讀操作，另外結合 Redis 等緩存來配合分擔數據庫的讀操作。

五、代碼層面進行讀寫分離

代碼環境是SpringBoot + SpringData JPA + Druib連接池。想要讀寫分離就需要配置多個數據源，在進行寫操作時選擇寫的數據源，讀操作時選擇讀的數據源。其中有兩個關鍵點：

1. 如何切換數據源
2. 如何根據不同的方法選擇正確的數據源

1)、如何切換數據源

通常用 SpringBoot 時都是使用它的默認配置，只需要在配置文件中定義好連接屬性就行了，但是現在我們需要自己來配置了，Spring 是支持多數據源的，多個datasource放在一個HashMapTargetDataSource中，通過dertermineCurrentLookupKey獲取key來決定要使用哪個數據源。因此我們的目標就很明確了，建立多個datasource放到TargetDataSource中，同時重寫dertermineCurrentLookupKey方法來決定使用哪個key。

用戶自定義設置數據庫路由

SpringBoot 提供了 AbstractRoutingDataSource 根據用戶定義的規則選擇當前的數據庫，這樣我們可以在執行查詢之前，設置讀取從庫，在執行完成後，恢復到主庫。AbstractRoutingDataSource 就是DataSource 的抽象，基於 lookupKey 的方式在多個數據庫中進行切換。重點關注setTargetDataSources，setDefaultTargetDataSource，determineCurrentLookupKey三個方法。那麼AbstractRoutingDataSource就是Spring讀寫分離的關鍵了。

實現可動態路由的數據源，在每次數據庫查詢操作前執行。

2)、如何選擇數據源

事務一般是註解在Service層的，因此在開始這個service方法調用時要確定數據源，有什麼通用方法能夠在開始執行一個方法前做操作呢？相信你已經想到了那就是切面 。怎麼切有兩種辦法：

註解式，定義一個只讀註解，被該數據標註的方法使用讀庫
方法名，根據方法名寫切點，比如getXXX用讀庫，setXXX用寫庫

六、部分代碼

DataSourcesConfig

/**
 * @author jingyu.bao
 * @version 1.0
 * @className DataSourceConfig
 * @description
 * @date 7/5/2020 20:09
 **/
@EnableTransactionManagement
@Configuration
public class DataSourceConfig {

    @Value("${spring.datasource.druid.master.name}")
    private String masterName;

    @Value("${spring.datasource.druid.master.url}")
    private String masterUrl;

    @Value("${spring.datasource.druid.master.username}")
    private String masterUsername;

    @Value("${spring.datasource.druid.master.password}")
    private String masterPassword;

    @Value("${spring.datasource.druid.master.driver-class-name}")
    private String masterDriverClassName;

    @Value("${spring.datasource.druid.slave.name}")
    private String slaveName;

    @Value("${spring.datasource.druid.slave.url}")
    private String slaveUrl;

    @Value("${spring.datasource.druid.slave.username}")
    private String slaveUsername;

    @Value("${spring.datasource.druid.slave.password}")
    private String slavePassword;

    @Value("${spring.datasource.druid.slave.driver-class-name}")
    private String slaveDriverClassName;

    @Value("${spring.datasource.druid.initial-size}")
    private String initialSize;

    @Value("${spring.datasource.druid.min-idle}")
    private String minIdle;

    @Value("${spring.datasource.druid.max-active}")
    private String maxActive;

    @Value("${spring.datasource.druid.max-wait}")
    private String maxWait;

    @Value("${spring.datasource.druid.time-between-eviction-runs-millis}")
    private String timeBetweenEvictionRunsMillis;

    @Value("${spring.datasource.druid.min-evictable-idle-time-millis}")
    private String minEvictableIdleTimeMillis;

    @Value("${spring.datasource.druid.validation-query}")
    private String validationQuery;

    @Value("${spring.datasource.druid.test-while-idle}")
    private String testWhileIdle;

    @Value("${spring.datasource.druid.test-on-borrow}")
    private String testOnBorrow;

    @Value("${spring.datasource.druid.test-on-return}")
    private String testOnReturn;

    @Value("${spring.datasource.druid.filters}")
    private String filters;

    @Value("{spring.datasource.druid.filter.stat.log-slow-sql}")
    private String logSlowSql;

    @Value("{spring.datasource.druid.filter.stat.slow-sql-millis}")
    private String slowSqlMillis;

    @Value("${spring.datasource.druid.type}")
    private String type;

    @Value("${spring.datasource.druid.stat-view-servlet.login-username}")
    private String loginUserName;

    @Value("${spring.datasource.druid.stat-view-servlet.login-password}")
    private String password;

    @Bean(name = "masterDataSource")
    public DataSource masterDataSource() {
        DruidDataSource datasource = new DruidDataSource();
        datasource.setUrl(masterUrl);
        datasource.setUsername(masterUsername);
        datasource.setPassword(masterPassword);
        datasource.setDriverClassName(masterDriverClassName);

        //configuration
        if (!StringUtils.isEmpty(initialSize)) {
            datasource.setInitialSize(Integer.parseInt(initialSize));
        }
        if (!StringUtils.isEmpty(minIdle)) {
            datasource.setMinIdle(Integer.parseInt(minIdle));
        }
        if (!StringUtils.isEmpty(maxActive)) {
            datasource.setMaxActive(Integer.parseInt(maxActive));
        }
        if (!StringUtils.isEmpty(maxWait)) {
            datasource.setMaxWait(Integer.parseInt(maxWait));
        }
        if (!StringUtils.isEmpty(timeBetweenEvictionRunsMillis)) {
            datasource.setTimeBetweenEvictionRunsMillis(Integer.parseInt(timeBetweenEvictionRunsMillis));
        }
        if (!StringUtils.isEmpty(minEvictableIdleTimeMillis)) {
            datasource.setMinEvictableIdleTimeMillis(Integer.parseInt(minEvictableIdleTimeMillis));
        }
        if (!StringUtils.isEmpty(validationQuery)) {
            datasource.setValidationQuery(validationQuery);
        }
        if (!StringUtils.isEmpty(testWhileIdle)) {
            datasource.setTestWhileIdle(Boolean.parseBoolean(testWhileIdle));
        }
        if (!StringUtils.isEmpty(testOnBorrow)) {
            datasource.setTestOnBorrow(Boolean.parseBoolean(testOnBorrow));
        }
        if (!StringUtils.isEmpty(testOnReturn)) {
            datasource.setTestOnReturn(Boolean.parseBoolean(testOnReturn));
        }

        try {
            datasource.setFilters(filters);
        } catch (SQLException e) {
            e.printStackTrace();
        }
        return datasource;
    }

    @Bean(name = "slaveDataSource")
    public DataSource slaveDataSource() {
        DruidDataSource datasource = new DruidDataSource();
        datasource.setUrl(masterUrl);
        datasource.setUsername(masterUsername);
        datasource.setPassword(masterPassword);
        datasource.setDriverClassName(masterDriverClassName);

        //configuration
        if (!StringUtils.isEmpty(initialSize)) {
            datasource.setInitialSize(Integer.parseInt(initialSize));
        }
        if (!StringUtils.isEmpty(minIdle)) {
            datasource.setMinIdle(Integer.parseInt(minIdle));
        }
        if (!StringUtils.isEmpty(maxActive)) {
            datasource.setMaxActive(Integer.parseInt(maxActive));
        }
        if (!StringUtils.isEmpty(maxWait)) {
            datasource.setMaxWait(Integer.parseInt(maxWait));
        }
        if (!StringUtils.isEmpty(timeBetweenEvictionRunsMillis)) {
            datasource.setTimeBetweenEvictionRunsMillis(Integer.parseInt(timeBetweenEvictionRunsMillis));
        }
        if (!StringUtils.isEmpty(minEvictableIdleTimeMillis)) {
            datasource.setMinEvictableIdleTimeMillis(Integer.parseInt(minEvictableIdleTimeMillis));
        }
        if (!StringUtils.isEmpty(validationQuery)) {
            datasource.setValidationQuery(validationQuery);
        }
        if (!StringUtils.isEmpty(testWhileIdle)) {
            datasource.setTestWhileIdle(Boolean.parseBoolean(testWhileIdle));
        }
        if (!StringUtils.isEmpty(testOnBorrow)) {
            datasource.setTestOnBorrow(Boolean.parseBoolean(testOnBorrow));
        }
        if (!StringUtils.isEmpty(testOnReturn)) {
            datasource.setTestOnReturn(Boolean.parseBoolean(testOnReturn));
        }

        try {
            datasource.setFilters(filters);
        } catch (SQLException e) {
            e.printStackTrace();
        }
        return datasource;
    }

    @Primary
    @Bean
    public DynamicRoutingDataSource dynamicDataSource(@Qualifier(value = "masterDataSource") DataSource masterDataSource, @Qualifier(value = "slaveDataSource") DataSource slaveDataSource) {
        Map<Object, Object> targetDataSources = new HashMap<>(2);
        targetDataSources.put(DynamicRoutingDataSourceContext.MASTER, masterDataSource);
        targetDataSources.put(DynamicRoutingDataSourceContext.SLAVE, slaveDataSource);
        DynamicRoutingDataSource dynamicRoutingDataSource = new DynamicRoutingDataSource();
        //設置數據源
        dynamicRoutingDataSource.setTargetDataSources(targetDataSources);
        //設置默認選擇的數據源
        dynamicRoutingDataSource.setDefaultTargetDataSource(masterDataSource);
        dynamicRoutingDataSource.afterPropertiesSet();
        return dynamicRoutingDataSource;
    }

    @Bean
    public ServletRegistrationBean statViewServlet() {
        ServletRegistrationBean servletRegistrationBean = new ServletRegistrationBean(new StatViewServlet(), "/druid/*");
        //設置ip白名單
        servletRegistrationBean.addInitParameter("allow", "");
        //設置ip黑名單，優先級高於白名單
        servletRegistrationBean.addInitParameter("deny", "");
        //設置控制檯管理用戶
        servletRegistrationBean.addInitParameter("loginUsername", loginUserName);
        servletRegistrationBean.addInitParameter("loginPassword", password);
        //是否可以重置數據
        servletRegistrationBean.addInitParameter("resetEnable", "false");
        return servletRegistrationBean;
    }

    @Bean
    public FilterRegistrationBean statFilter() {
        //創建過濾器
        FilterRegistrationBean filterRegistrationBean = new FilterRegistrationBean(new WebStatFilter());
        //設置過濾器過濾路徑
        filterRegistrationBean.addUrlPatterns("/*");
        //忽略過濾的形式
        filterRegistrationBean.addInitParameter("exclusions", "*.js,*.gif,*.jpg,*.png,*.css,*.ico,/druid/*");
        return filterRegistrationBean;
    }

DynamicRoutingDataSource

/**
 * @author jingyu.bao
 * @version 1.0
 * @className DynamicRoutingDataSource
 * @description
 * @date 7/5/2020 21:22
 **/
public class DynamicRoutingDataSource extends AbstractRoutingDataSource {
    @Override
    protected Object determineCurrentLookupKey() {
        Object lookupKey = DynamicRoutingDataSourceContext.getRoutingDataSource();
        System.err.println(Thread.currentThread().getName() + " determineCurrentLookupKey : " + lookupKey);
        return lookupKey;
    }
}

DynamicRoutingDataSourceContext

/**
 * @author jingyu.bao
 * @version 1.0
 * @className DynamicRoutingDataSourceContext
 * @description
 * @date 7/5/2020 20:16
 **/
public class DynamicRoutingDataSourceContext {

    public static final String MASTER = "master";

    public static final String SLAVE = "slave";

    private static final ThreadLocal<Object> threadLocalDataSource = new ThreadLocal<>();


    public static void setRoutingDataSource(Object dataSource) {
        if (dataSource == null) {
            throw new NullPointerException();
        }
        threadLocalDataSource.set(dataSource);
        // System.err.println(Thread.currentThread().getName()+" set RoutingDataSource : " + dataSource);
    }

    public static Object getRoutingDataSource() {
        Object dataSourceType = threadLocalDataSource.get();
        if (dataSourceType == null) {
            threadLocalDataSource.set(DynamicRoutingDataSourceContext.MASTER);
            return getRoutingDataSource();
        }
        // System.err.println(Thread.currentThread().getName()+" get RoutingDataSource : " + dataSourceType);
        return dataSourceType;
    }

    public static void removeRoutingDataSource() {
        threadLocalDataSource.remove();
        // System.err.println(Thread.currentThread().getName()+" remove RoutingDataSource");
    }
}

RoutingAopAspect

/**
 * @author jingyu.bao
 * @version 1.0
 * @className RoutingAopAspect
 * @description
 * @date 7/5/2020 20:21
 **/
@Order(0)
@Aspect
@Component
public class RoutingAopAspect {

    @Around("@annotation(targetDataSource)")
    public Object routingWithDataSource(ProceedingJoinPoint joinPoint, TargetDataSource targetDataSource) throws Throwable {
        try {
            DynamicRoutingDataSourceContext.setRoutingDataSource(targetDataSource.value());
            return joinPoint.proceed();
        } finally {
            DynamicRoutingDataSourceContext.removeRoutingDataSource();
        }
    }
}

TargetDataSource

/**
 * @author jingyu.bao
 * @version 1.0
 * @className TargetDataSource
 * @description
 * @date 7/5/2020 20:40
 **/
@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(value = RetentionPolicy.RUNTIME)
@Documented
public @interface TargetDataSource {
    String value();
}

UserInfoServiceImpl

/**
 * @author jingyu.bao
 * @version 1.0
 * @className UserServiceImpl
 * @description
 * @date 7/5/2020 21:39
 **/
@Service
public class UserInfoServiceImpl implements IUserInfoService {

    @Autowired
    private IUserInfoRepository userInfoRepository;

    @TargetDataSource(value = "slave")
    @Override
    public List<UserInfoEntity> findAll() {
        return userInfoRepository.findAll();
    }

    @Transactional
    @Override
    public UserInfoEntity save(UserInfoEntity userInfoEntity) {
        return userInfoRepository.save(userInfoEntity);
    }

    @TargetDataSource(value = "slave")
    @Override
    public UserInfoEntity findById(Long id) {
        Optional<UserInfoEntity> userInfoEntity = userInfoRepository.findById(id);
        return userInfoEntity.isPresent() ? userInfoEntity.get() : null;
    }

    @Override
    public List<UserInfoEntity> findAllMaster() {
        return userInfoRepository.findAll();
    }

    @Transactional
    @Override
    public void saveAll(ArrayList<UserInfoEntity> list) {
        userInfoRepository.saveAll(list);
    }
}

application.properties

server.port=8080
server.tomcat.max-threads=3000
server.tomcat.max-connections=20000
server.tomcat.uri-encoding=UTF-8
server.tomcat.accept-count=800
# 自定義線程池參數
fxea.threadPool.coreThreadNum=5
fxea.threadPool.maxThreadNum=25
# 這個參數是在建表的時候，將默認的存儲引擎切換爲 InnoDB 用的
spring.jpa.database-platform=org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.database=mysql
# 配置在日誌中打印出執行的 SQL 語句信息。
spring.jpa.show-sql=true
# 第一次建表create,後面用update,要不然每次重啓都會新建表
spring.jpa.hibernate.ddl-auto=create
# Druid
spring.datasource.druid.type=com.alibaba.druid.pool.DruidDataSource
#初始化連接大小
spring.datasource.druid.initial-size=10
#最小連接池數量
spring.datasource.druid.min-idle=10
#最大連接池數量
spring.datasource.druid.max-active=100
#配置獲取連接等待超時的時間
spring.datasource.druid.max-wait=60000
#配置間隔多久才進行一次檢測，檢測需要關閉的空閒連接，單位是毫秒
spring.datasource.druid.time-between-eviction-runs-millis=60000
#配置一個連接在池中最小生存的時間，單位是毫秒
spring.datasource.druid.min-evictable-idle-time-millis=300000
#測試連接
spring.datasource.druid.validation-query=SELECT 'x'
#申請連接的時候檢測，建議配置爲true，不影響性能，並且保證安全性
spring.datasource.druid.test-while-idle=true
#獲取連接時執行檢測，建議關閉，影響性能
spring.datasource.druid.test-on-borrow=false
#歸還連接時執行檢測，建議關閉，影響性能
spring.datasource.druid.test-on-return=false
#druid 用戶
spring.datasource.druid.stat-view-servlet.login-username=admin
#druid 密碼
spring.datasource.druid.stat-view-servlet.login-password=admin
#配置監控統計攔截的filters，去掉後監控界面sql無法統計，'wall'用於防火牆
spring.datasource.druid.filters=stat,wall
spring.datasource.druid.filter.stat.log-slow-sql=true
spring.datasource.druid.filter.stat.slow-sql-millis=1
# Db Master
spring.datasource.druid.master.name=master
spring.datasource.druid.master.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.druid.master.url=jdbc:mysql://localhost:3306/test?useSSL=false&characterEncoding=UTF-8&autoReconnect=true&zeroDateTimeBehavior=convertToNull&useUnicode=true
spring.datasource.druid.master.username=root
spring.datasource.druid.master.password=123456
# Db Slaves
spring.datasource.druid.slave.name=slave
spring.datasource.druid.slave.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.druid.slave.url=jdbc:mysql://localhost:3307/test?useSSL=false&characterEncoding=UTF-8&autoReconnect=true&zeroDateTimeBehavior=convertToNull&useUnicode=true
spring.datasource.druid.slave.username=root
spring.datasource.druid.slave.password=123456

項目地址：https://github.com/baojingyu/spring-boot-jpa-dynamic-datasource

點個關注，和我一起共同進步吧！