SpringBoot數據庫讀寫分離

背景

一個項目中數據庫最基礎同時也是最主流的是單機數據庫，讀寫都在一個庫中。當用戶逐漸增多，單機數據庫無法滿足性能要求時，就會進行讀寫分離改造（適用於讀多寫少），寫操作一個庫，讀操作多個庫，通常會做一個數據庫集羣，開啓主從備份，一主多從，以提高讀取性能。當用戶更多讀寫分離也無法滿足時，就需要分佈式數據庫了-NoSQL。
正常情況下讀寫分離的實現，首先要做一個一主多從的數據庫集羣，同時還需要進行數據同步。

數據庫主從搭建

Master配置

①修改/etc/my.cnf

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

server-id=1
log-bin=mysql-bin
binlog-do-db=zpark
binlog-do-db=baizhi
binlog-ignore-db=mysql
binlog-ignore-db=test
expire_logs_days=10
auto_increment_increment=2
auto_increment_offset=1

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

②重啓MySQL服務

[root@CentOS ~]# service mysqld restart
Stopping mysqld:                                           [  OK  ]
Starting mysqld:                                           [  OK  ]

③登錄MySQL主機查看狀態

[root@CentOS ~]# mysql -u root -proot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.1.73-log Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show master status\G
*************************** 1. row ***************************
            File: mysql-bin.000001
        Position: 106
    Binlog_Do_DB: zpark,baizhi
Binlog_Ignore_DB: mysql,test
1 row in set (0.00 sec)

Slave 配置

① 修改/etc/my.cnf文件

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

server-id=2
log-bin=mysql-bin
replicate-do-db=zpark
replicate-do-db=baizhi
replicate-ignore-db=mysql
replicate-ignore-db=test
expire_logs_days=10
auto_increment_increment=2
auto_increment_offset=2

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

③ 重啓MySQL服務

[root@CentOS ~]# service mysqld restart
Stopping mysqld:                                           [  OK  ]
Starting mysqld:                                           [  OK  ]

④ MySQL配置從機

[root@CentOS ~]# mysql -u root -proot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.1.73-log Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> stop slave;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> change master to master_host='主機IP',master_user='root',master_password='root',master_log_file='mysql-bin.000001',master_log_pos=106;
Query OK, 0 rows affected (0.03 sec)

mysql> start slave;
Query OK, 0 rows affected (0.01 sec)

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: CentOSB
                  Master_User: root
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000001
          Read_Master_Log_Pos: 106
               Relay_Log_File: mysqld-relay-bin.000002
                Relay_Log_Pos: 251
        Relay_Master_Log_File: mysql-bin.000001
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: zpark,baizhi
          Replicate_Ignore_DB: mysql,test
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 106
              Relay_Log_Space: 407
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
1 row in set (0.00 sec)

讀寫分離實現

讀寫分離要做的事情就是對於一條SQL該選擇哪個數據庫去執行，至於誰來做選擇數據庫這件事兒，無非兩個，要麼中間件幫我們做，要麼程序自己做。因此，一般來講，讀寫分離有兩種實現方式。第一種是依靠中間件（比如：MyCat），也就是說應用程序連接到中間件，中間件幫我們做SQL分離；第二種是應用程序自己去做分離。

編碼思想

所謂的手寫讀寫分離，需要用戶自定義一個動態的數據源，該數據源可以根據當前上下文中調用方法是讀或者是寫方法決定返回主庫的鏈接還是從庫的鏈接。這裏我們使用Spring提供的一個代理數據源AbstractRoutingDataSource接口。

該接口需要用戶完善一個determineCurrentLookupKey抽象法，系統會根據這個抽象返回值決定使用系統中定義的數據源。

@Nullable
protected abstract Object determineCurrentLookupKey();

其次該類還有兩個屬性需要指定defaultTargetDataSource和targetDataSources，其中defaultTargetDataSource需要指定爲Master數據源。targetDataSources是一個Map需要將所有的數據源添加到該Map中，以後系統會根據determineCurrentLookupKey方法的返回值作爲key從targetDataSources查找相應的實際數據源。如果找不到則使用defaultTargetDataSource指定的數據源。

實現步驟

①添加依賴

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.1.5.RELEASE</version>
</parent>
<!--SpringWeb Test-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!--文件支持-->
<dependency>
    <groupId>commons-fileupload</groupId>
    <artifactId>commons-fileupload</artifactId>
    <version>1.4</version>
</dependency>

<!--Junit測試-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
</dependency>

<!--MySQL & MyBatis-->
<dependency>
    <groupId>org.mybatis.spring.boot</groupId>
    <artifactId>mybatis-spring-boot-starter</artifactId>
    <version>2.0.1</version>
</dependency>
<dependency>
    <groupId>mysql</groupId>
    <artifactId>mysql-connector-java</artifactId>
    <version>5.1.47</version>
</dependency>

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

②application.properties

server.port=8888
server.servlet.context-path=/

spring.servlet.multipart.max-file-size=1MB
spring.servlet.multipart.enabled=true

# spring.servlet.multipart.location=E:/uploadfiles
#spring.datasource.username=root
#spring.datasource.password=root
#spring.datasource.driver-class-name=com.mysql.jdbc.Driver
#spring.datasource.url=jdbc:mysql://localhost:3306/test?useUnicode=true&characterEncoding=UTF8&serverTimezone=UTC&useSSL=false

spring.datasource.master.username=root
spring.datasource.master.password=root
spring.datasource.master.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.master.jdbc-url=jdbc:mysql://CentOSB:3306/baizhi?useUnicode=true&characterEncoding=UTF8&serverTimezone=UTC&useSSL=false

spring.datasource.slave1.username=root
spring.datasource.slave1.password=root
spring.datasource.slave1.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.slave1.jdbc-url=jdbc:mysql://CentOSC:3306/baizhi?useUnicode=true&characterEncoding=UTF8&serverTimezone=UTC&useSSL=false

spring.datasource.slave2.username=root
spring.datasource.slave2.password=root
spring.datasource.slave2.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.slave2.jdbc-url=jdbc:mysql://CentOSC:3306/baizhi?useUnicode=true&characterEncoding=UTF8&serverTimezone=UTC&useSSL=false

## MyBatis DAO
#mybatis.type-aliases-package=com.baizhi.entities
#mybatis.mapper-locations=classpath*:mappers/*.xml
#mybatis.executor-type=batch

spring.http.encoding.charset=utf-8
spring.jackson.time-zone=GMT+8

③配置數據源

/**
 * 該類是自定義數據源，由於必須將系統的數據源給替換掉。
 */
@Configuration
public class UserDefineDatasourceConfig {

    @Bean
    @ConfigurationProperties("spring.datasource.master")
    public DataSource masterDataSource() {
        return DataSourceBuilder.create().build();
    }

    @Bean
    @ConfigurationProperties("spring.datasource.slave1")
    public DataSource slave1DataSource() {
        return DataSourceBuilder.create().build();
    }

    @Bean
    @ConfigurationProperties("spring.datasource.slave2")
    public DataSource slave2DataSource() {
        return DataSourceBuilder.create().build();
    }

    @Bean
    public DataSource userDefineRoutingDataSource(@Qualifier("masterDataSource") DataSource masterDataSource,
                                          @Qualifier("slave1DataSource") DataSource slave1DataSource,
                                          @Qualifier("slave2DataSource") DataSource slave2DataSource) {

        Map<Object, Object> targetDataSources = new HashMap<>();
        targetDataSources.put("master", masterDataSource);
        targetDataSources.put("slave-01", slave1DataSource);
        targetDataSources.put("slave-02", slave2DataSource);

        List<String> slaveDBKeys= Arrays.asList("slave-01","slave-02");

        DynamicRoutingDataSource userDefineRoutingDataSource = new DynamicRoutingDataSource(slaveDBKeys);
        //設置默認數據源
        userDefineRoutingDataSource.setDefaultTargetDataSource(masterDataSource);
        //設定主數據源
        userDefineRoutingDataSource.setTargetDataSources(targetDataSources);

        return userDefineRoutingDataSource;
    }


    /**
     * 當自定義數據源，用戶必須覆蓋SqlSessionFactory創建
     * @param dataSource
     * @return
     * @throws Exception
     */
    @Bean
    public SqlSessionFactory sqlSessionFactory(@Qualifier("userDefineRoutingDataSource") DataSource dataSource) throws Exception {
        SqlSessionFactoryBean sqlSessionFactoryBean = new SqlSessionFactoryBean();

        sqlSessionFactoryBean.setDataSource(dataSource);
        sqlSessionFactoryBean.setTypeAliasesPackage("com.baizhi.entities");
        sqlSessionFactoryBean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath*:mappers/*.xml"));

        SqlSessionFactory sqlSessionFactory = sqlSessionFactoryBean.getObject();

        return sqlSessionFactory;
    }

    /**
     * 當自定義數據源，用戶必須覆蓋SqlSessionTemplate,開啓BATCH處理模式
     * @param sqlSessionFactory
     * @return
     */
    @Bean
    public SqlSessionTemplate sqlSessionTemplate(@Qualifier("sqlSessionFactory") SqlSessionFactory sqlSessionFactory) {
        ExecutorType executorType = ExecutorType.BATCH;
        if (executorType != null) {
            return new SqlSessionTemplate(sqlSessionFactory, executorType);
        } else {
            return new SqlSessionTemplate(sqlSessionFactory);
        }
    }

    /***
     * 當自定義數據源，用戶必須注入，否則事務控制不生效
     * @param dataSource
     * @return
     */
    @Bean
    public PlatformTransactionManager platformTransactionManager(@Qualifier("userDefineRoutingDataSource") DataSource dataSource) {
        return new DataSourceTransactionManager(dataSource);
    }
}

④配置切面

/**
 * 用戶自定義切面，負責讀取SlaveDB註解，並且在DBTypeContextHolder中設置讀寫類型
 */
@Aspect
@Order(0) //必須設置0，表示Spring會將UserDefineDataSourceAOP放置在所有切面的前面
@Component
public class UserDefineDataSourceAOP  {
    private static final Logger logger = LoggerFactory.getLogger(UserDefineDataSourceAOP.class);

    /**
     * 設置環繞切面，該切面的作用是設置當前上下文的讀寫類型
     * @param pjp
     * @return
     */
    @Around("execution(* com.baizhi.service..*.*(..))")
    public Object around(ProceedingJoinPoint pjp) {
       //獲取方法對象
        MethodSignature signature = (MethodSignature) pjp.getSignature();
        Method method = signature.getMethod();
        Object result = null;
        try {
            //獲取方法上註解，設置讀寫模式
            if(method.isAnnotationPresent(SlaveDB.class)){
                DBTypeContextHolder.set(OperatorTypeEnum.READ);
            }else{
                DBTypeContextHolder.set(OperatorTypeEnum.WRITE);
            }
            logger.debug("設置操作類型："+DBTypeContextHolder.get());
            result = pjp.proceed();
            //清除狀態
            DBTypeContextHolder.clear();
        } catch (Throwable throwable) {
          throw new RuntimeException(throwable.getCause());
        }

        return result;
    }
}

⑤用到的其他類

• 動態數據源

/**
 * 該類屬於代理數據源，負責負載均衡
 */
public class DynamicRoutingDataSource extends AbstractRoutingDataSource {
    private static final Logger logger = LoggerFactory.getLogger(DynamicRoutingDataSource.class);

    private List<String> slaveDataSourceKey=new  ArrayList();
    private String masterKey="master";
    private static AtomicInteger round=new AtomicInteger(0);

    public DynamicRoutingDataSource(List<String> slaveDataSourceKey) {
        this.slaveDataSourceKey = slaveDataSourceKey;
    }

    /**
     * 系統會根據 該方法的返回值，決定使用定義的那種數據源
     * @return
     */
    @Override
    protected Object determineCurrentLookupKey() {
        String currentDBKey="";
        OperatorTypeEnum operatorType=DBTypeContextHolder.get();
        if(operatorType.equals(OperatorTypeEnum.WRITE)){
            currentDBKey= masterKey;
        }else{
            int value = round.incrementAndGet();
            if(value<0){
                round.set(0);
            }
            currentDBKey= slaveDataSourceKey.get(round.get()%slaveDataSourceKey.size());
        }
        logger.debug("當前使用的操作:"+operatorType+",使用DBKey"+currentDBKey);
        return currentDBKey;
    }
}

• 讀寫類型

/**
 * 寫、讀類型
 */
public enum OperatorTypeEnum {
    WRITE, READ;
}

• 記錄操作類型

/**
 * 該類主要是用於存儲，當前用戶的操作類型，將當前的操作存儲在當前線程的上下文中
 */
public class OPTypeContextHolder {
    private static final ThreadLocal<OperatorTypeEnum> OPERATOR_TYPE_THREAD_LOCAL = new ThreadLocal<>();
    public static void set(OperatorTypeEnum dbType) {
        OPERATOR_TYPE_THREAD_LOCAL.set(dbType);
    }

    public static OperatorTypeEnum get() {
        return OPERATOR_TYPE_THREAD_LOCAL.get();
    }
    public static void clear(){
        OPERATOR_TYPE_THREAD_LOCAL.remove();
    }
}

• 業務方法標記註解

/**
 * 該註解用於標註，當前用戶的調用方法是讀還是寫
 */
@Retention(RetentionPolicy.RUNTIME) //表示運行時解析註解
@Target(value = {ElementType.METHOD})//表示只能在方法上加
public @interface SlaveDB { }

附錄 logback.xml配置

<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="60 seconds" debug="false">

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender" >
        <encoder>
            <pattern>%p %c#%M %d{yyyy-MM-dd HH:mm:ss} %m%n</pattern>
            <charset>UTF-8</charset>
        </encoder>
    </appender>
    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
　　　　　　<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
　　　　　　　　　<fileNamePattern>logs/userLoginFile-%d{yyyyMMdd}.log</fileNamePattern>
　　　　　　　　　<maxHistory>30</maxHistory>
　　　　　　</rollingPolicy>
　　　　　　<encoder>
　　　　　　　　　<pattern>%p %c#%M %d{yyyy-MM-dd HH:mm:ss} %m%n</pattern>
                 <charset>UTF-8</charset>
　　　　　　</encoder>
    </appender>

    <!-- 控制檯輸出日誌級別 -->
    <root level="ERROR">
        <appender-ref ref="STDOUT" />
    </root>

    <logger name="org.springframework.jdbc" level="DEBUG" additivity="false">
        <appender-ref ref="STDOUT" />
    </logger>
    <logger name="com.baizhi.dao" level="TRACE" additivity="false">
        <appender-ref ref="STDOUT" />
    </logger>
    <logger name="com.baizhi.cache" level="DEBUG" additivity="false">
        <appender-ref ref="STDOUT" />
    </logger>
    <logger name="com.baizhi.datasource" level="DEBUG" additivity="false">
        <appender-ref ref="STDOUT" />
    </logger>
</configuration>