Java猿社區—ShardingSphere-4.0.1之實現分庫分表+讀寫分離
文章目錄
參考：
基於Docker的Mysql主從複製搭建
ShardingSphere官網
技術體系
Springboot2.1.5 + shardingsphere4.0.1 + Mysql5.7 + mysql-connector-java5.1.47
背景
目前公司正在進行的在線客服IM項目，考慮未來會話消息請求流量劇增以及消息存儲帶來的挑戰，單一數據庫無法支撐，需要分庫&分表、讀寫分離以應對高併發帶來的挑戰。
基於市面上較爲流行的幾個數據庫中間件我們結合公司技術體系，再三考慮，決定使用ShardingSphere作爲在線客服項目的數據庫中間件。
ShardingSphere介紹
Apache ShardingSphere(Incubator) 是一套開源的分佈式數據庫中間件解決方案組成的生態圈，它由Sharding-JDBC、Sharding-Proxy和Sharding-Sidecar（規劃中）這3款相互獨立，卻又能夠混合部署配合使用的產品組成。它們均提供標準化的數據分片、分佈式事務和數據庫治理功能，可適用於如Java同構、異構語言、雲原生等各種多樣化的應用場景。（來自官網）
ShardingShpere支持的功能
數據分片
- 分庫、分表
- 讀寫分離
- 分片策略定製化
- 無中心化分佈式主鍵
分佈式事務
- 標準化事務接口
- XA強一致事務
- 柔性事務
技術準備
mysql安裝配置
具體安裝配置請參見網絡博客：如基於Docker的Mysql主從複製搭建
POM配置
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.zzx.shardingsphere</groupId>
<artifactId>zzx-shardingsphere</artifactId>
<packaging>pom</packaging>
<version>1.0-SNAPSHOT</version>
<modules>
<module>db-read-write</module>
<module>db-table-read-write</module>
</modules>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.5.RELEASE</version>
<relativePath/>
</parent>
<properties>
<java.version>1.8</java.version>
<mybatis-spring-boot>2.0.1</mybatis-spring-boot>
<druid>1.1.16</druid>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<shardingsphere.version>4.0.1</shardingsphere.version>
<fastjson.version>1.2.38</fastjson.version>
<mysql-connector-java.version>5.1.47</mysql-connector-java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>${mybatis-spring-boot}</version>
</dependency>
<!--mybatis驅動-->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${mysql-connector-java.version}</version>
</dependency>
<!--druid數據源-->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid-spring-boot-starter</artifactId>
<version>${druid}</version>
</dependency>
<!--shardingsphere版本-->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-namespace</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<!--lombok實體工具-->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>${fastjson.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
分庫分表+讀寫分離
mysql配置環境
主庫0： localhost:3342/im_bis
主庫0->從庫0: localhost:3343/im_bis
主庫1: localhost:3344/im_bis
主庫1->從庫1: localhost:3345/im_bis
配置從庫0和從庫1分別作爲主庫0和主庫1的從庫，具體配置參考上面提到的基於Docker搭建主從複製網站
sql腳本
/*
Navicat Premium Data Transfer
Source Server : 192.168.89.53
Source Server Type : MySQL
Source Server Version : 50726
Source Host : 192.168.89.53:3306
Source Schema : ddky_im_bis
Target Server Type : MySQL
Target Server Version : 50726
File Encoding : 65001
Date: 08/03/2020 13:18:39
*/
DROP SCHEMA IF EXISTS im_bis;
CREATE SCHEMA IF NOT EXISTS im_bis;
use im_bis
SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for im_team_message
-- ----------------------------
DROP TABLE IF EXISTS `im_team_message0`;
CREATE TABLE `im_team_message0` (
`id` bigint(20) NOT NULL COMMENT '主鍵id',
`team_msg_id` bigint(20) DEFAULT NULL COMMENT '羣消息id(服務端生成消息id)',
`team_client_msg_id` varchar(32) DEFAULT '' COMMENT '羣消息id（客戶端生成）',
`team_id` bigint(20) DEFAULT NULL COMMENT '羣組id（對應羣組的accid）',
`msg_type` varchar(15) DEFAULT 'TEXT' COMMENT '消息類型：TEXT-文本，PICTURE-圖片，AUDIO-音頻，VIDEO-視頻，FILE-文件，GEO-地理位置，CUSTOM-自定義，TIP-提醒，ROBOT-AI機器人，NOTICATION-羣通知，TEAM_INVITE-邀請入羣，TEAM_INVITE_REJECT-拒絕邀請，CUSTOM_TEAM_MSG -羣組自定義系統通知',
`event_type` tinyint(4) DEFAULT '1' COMMENT '事件類型：1-會話類型消息（p2p消息、羣聊消息、自定義系統通知、雲信內置系統通知），2-登錄事件，3-登出/離線事件，4-聊天室聊天消息，5-音視頻時長、白板時長消息，6-音視頻白板大小、下載地址消息，7-單聊消息撤回，8-羣聊消息撤回，9-彙報主播、管理員進出聊天室事件消息，10-彙報專線電話通話結束回調抄送的消息，11-彙報短信回執抄送的消息，12-彙報短信上行消息，13-彙報用戶進出音視頻/白板房間的消息，14-彙報聊天室隊列操作的事件消息，20-易盾異步反垃圾結果信息',
`remind_type` tinyint(4) DEFAULT '0' COMMENT '提醒類型：0-普通消息，1-客服進入，2-客戶進入，3-客服進入歡迎提醒，4-敏感詞命中提醒消息',
`conv_type` varchar(20) DEFAULT '' COMMENT '場景類型：TEAM',
`scene` tinyint(4) DEFAULT '0' COMMENT '場景：0-team，1-p2p，2-superTeam',
`text` varchar(5000) DEFAULT '' COMMENT '內容',
`attach` varchar(5000) DEFAULT '' COMMENT '附加消息',
`team_attach_type` tinyint(4) DEFAULT NULL COMMENT '羣通知類型：默認null（普通消息），0-更新羣，1-拉人入羣，2-踢人出羣，3-接受入羣邀請，4-通過入羣邀請，5-添加羣管理員，6-移除羣管理員，7-主動退羣，8-解散羣，9-轉讓羣，10-更新羣成員禁言狀態',
`send_type` tinyint(4) DEFAULT '0' COMMENT '發送人類型：0-用戶，1-客服，2-藥師，3-醫生',
`send_client_ip` varchar(20) DEFAULT '' COMMENT '發送端ip',
`send_client_port` varchar(5) DEFAULT '' COMMENT '發送端端口',
`send_client_type` varchar(10) DEFAULT '' COMMENT '發送客戶端類型： AOS、IOS、PC、WINPHONE、WEB、REST',
`send_device_id` varchar(32) DEFAULT '' COMMENT '發送端設備編號',
`send_nick` varchar(50) DEFAULT '' COMMENT '發送人暱稱',
`send_id` varchar(32) DEFAULT '' COMMENT '發送人id（對應客服表和用戶表的accid）',
`send_at` bigint(11) DEFAULT NULL COMMENT '發送時間',
`msg_receipt_time` bigint(11) DEFAULT NULL COMMENT '已讀回執時間戳，如果有此字段, 說明此時間戳之前的所有消息對方均已讀',
`is_revoke` tinyint(4) DEFAULT '0' COMMENT '是否撤銷：0-否，1-是',
`revoke_at` bigint(11) DEFAULT '0' COMMENT '撤銷時間',
`custom_apns_text` varchar(200) DEFAULT '' COMMENT '自定義系統通知消息推送文本。僅在convType爲CUSTOM_PERSON或CUSTOM_TEAM時含此字段',
`ext` varchar(64) DEFAULT '' COMMENT '消息擴展字段',
`antispam` varchar(12) DEFAULT '' COMMENT '標識是否被反垃圾(‘’，true，false)',
`yidun_res` varchar(500) DEFAULT '' COMMENT '易盾反垃圾的原始處理細節',
`msg_status` tinyint(4) DEFAULT '0' COMMENT '消息發送狀態：0-發送成功，1-發送中，2-發送失敗',
`status` tinyint(4) DEFAULT '0' COMMENT '狀態：有效 0：無效',
PRIMARY KEY (`id`),
UNIQUE KEY `index_tt` (`id`,`team_msg_id`,`team_client_msg_id`) USING BTREE,
KEY `index_team_id_on_im_team_message` (`team_id`),
KEY `index_send_at_on_im_team_message` (`send_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='IM消息表';
SET FOREIGN_KEY_CHECKS = 1;
/*
Navicat Premium Data Transfer
Source Server : 192.168.89.53
Source Server Type : MySQL
Source Server Version : 50726
Source Host : 192.168.89.53:3306
Source Schema : ddky_im_bis
Target Server Type : MySQL
Target Server Version : 50726
File Encoding : 65001
Date: 08/03/2020 13:18:39
*/
SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for im_team_message
-- ----------------------------
DROP TABLE IF EXISTS `im_team_message1`;
CREATE TABLE `im_team_message1` (
`id` bigint(20) NOT NULL COMMENT '主鍵id',
`team_msg_id` bigint(20) DEFAULT NULL COMMENT '羣消息id(服務端生成消息id)',
`team_client_msg_id` varchar(32) DEFAULT '' COMMENT '羣消息id（客戶端生成）',
`team_id` bigint(20) DEFAULT NULL COMMENT '羣組id（對應羣組的accid）',
`msg_type` varchar(15) DEFAULT 'TEXT' COMMENT '消息類型：TEXT-文本，PICTURE-圖片，AUDIO-音頻，VIDEO-視頻，FILE-文件，GEO-地理位置，CUSTOM-自定義，TIP-提醒，ROBOT-AI機器人，NOTICATION-羣通知，TEAM_INVITE-邀請入羣，TEAM_INVITE_REJECT-拒絕邀請，CUSTOM_TEAM_MSG -羣組自定義系統通知',
`event_type` tinyint(4) DEFAULT '1' COMMENT '事件類型：1-會話類型消息（p2p消息、羣聊消息、自定義系統通知、雲信內置系統通知），2-登錄事件，3-登出/離線事件，4-聊天室聊天消息，5-音視頻時長、白板時長消息，6-音視頻白板大小、下載地址消息，7-單聊消息撤回，8-羣聊消息撤回，9-彙報主播、管理員進出聊天室事件消息，10-彙報專線電話通話結束回調抄送的消息，11-彙報短信回執抄送的消息，12-彙報短信上行消息，13-彙報用戶進出音視頻/白板房間的消息，14-彙報聊天室隊列操作的事件消息，20-易盾異步反垃圾結果信息',
`remind_type` tinyint(4) DEFAULT '0' COMMENT '提醒類型：0-普通消息，1-客服進入，2-客戶進入，3-客服進入歡迎提醒，4-敏感詞命中提醒消息',
`conv_type` varchar(20) DEFAULT '' COMMENT '場景類型：TEAM',
`scene` tinyint(4) DEFAULT '0' COMMENT '場景：0-team，1-p2p，2-superTeam',
`text` varchar(5000) DEFAULT '' COMMENT '內容',
`attach` varchar(5000) DEFAULT '' COMMENT '附加消息',
`team_attach_type` tinyint(4) DEFAULT NULL COMMENT '羣通知類型：默認null（普通消息），0-更新羣，1-拉人入羣，2-踢人出羣，3-接受入羣邀請，4-通過入羣邀請，5-添加羣管理員，6-移除羣管理員，7-主動退羣，8-解散羣，9-轉讓羣，10-更新羣成員禁言狀態',
`send_type` tinyint(4) DEFAULT '0' COMMENT '發送人類型：0-用戶，1-客服，2-藥師，3-醫生',
`send_client_ip` varchar(20) DEFAULT '' COMMENT '發送端ip',
`send_client_port` varchar(5) DEFAULT '' COMMENT '發送端端口',
`send_client_type` varchar(10) DEFAULT '' COMMENT '發送客戶端類型： AOS、IOS、PC、WINPHONE、WEB、REST',
`send_device_id` varchar(32) DEFAULT '' COMMENT '發送端設備編號',
`send_nick` varchar(50) DEFAULT '' COMMENT '發送人暱稱',
`send_id` varchar(32) DEFAULT '' COMMENT '發送人id（對應客服表和用戶表的accid）',
`send_at` bigint(11) DEFAULT NULL COMMENT '發送時間',
`msg_receipt_time` bigint(11) DEFAULT NULL COMMENT '已讀回執時間戳，如果有此字段, 說明此時間戳之前的所有消息對方均已讀',
`is_revoke` tinyint(4) DEFAULT '0' COMMENT '是否撤銷：0-否，1-是',
`revoke_at` bigint(11) DEFAULT '0' COMMENT '撤銷時間',
`custom_apns_text` varchar(200) DEFAULT '' COMMENT '自定義系統通知消息推送文本。僅在convType爲CUSTOM_PERSON或CUSTOM_TEAM時含此字段',
`ext` varchar(64) DEFAULT '' COMMENT '消息擴展字段',
`antispam` varchar(12) DEFAULT '' COMMENT '標識是否被反垃圾(‘’，true，false)',
`yidun_res` varchar(500) DEFAULT '' COMMENT '易盾反垃圾的原始處理細節',
`msg_status` tinyint(4) DEFAULT '0' COMMENT '消息發送狀態：0-發送成功，1-發送中，2-發送失敗',
`status` tinyint(4) DEFAULT '0' COMMENT '狀態：有效 0：無效',
PRIMARY KEY (`id`),
UNIQUE KEY `index_tt` (`id`,`team_msg_id`,`team_client_msg_id`) USING BTREE,
KEY `index_team_id_on_im_team_message` (`team_id`),
KEY `index_send_at_on_im_team_message` (`send_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='IM消息表';
SET FOREIGN_KEY_CHECKS = 1;
/*
Navicat Premium Data Transfer
Source Server : 192.168.89.53
Source Server Type : MySQL
Source Server Version : 50726
Source Host : 192.168.89.53:3306
Source Schema : ddky_im_bis
Target Server Type : MySQL
Target Server Version : 50726
File Encoding : 65001
Date: 08/03/2020 13:18:39
*/
SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for im_team_message
-- ----------------------------
DROP TABLE IF EXISTS `im_team_message2`;
CREATE TABLE `im_team_message2` (
`id` bigint(20) NOT NULL COMMENT '主鍵id',
`team_msg_id` bigint(20) DEFAULT NULL COMMENT '羣消息id(服務端生成消息id)',
`team_client_msg_id` varchar(32) DEFAULT '' COMMENT '羣消息id（客戶端生成）',
`team_id` bigint(20) DEFAULT NULL COMMENT '羣組id（對應羣組的accid）',
`msg_type` varchar(15) DEFAULT 'TEXT' COMMENT '消息類型：TEXT-文本，PICTURE-圖片，AUDIO-音頻，VIDEO-視頻，FILE-文件，GEO-地理位置，CUSTOM-自定義，TIP-提醒，ROBOT-AI機器人，NOTICATION-羣通知，TEAM_INVITE-邀請入羣，TEAM_INVITE_REJECT-拒絕邀請，CUSTOM_TEAM_MSG -羣組自定義系統通知',
`event_type` tinyint(4) DEFAULT '1' COMMENT '事件類型：1-會話類型消息（p2p消息、羣聊消息、自定義系統通知、雲信內置系統通知），2-登錄事件，3-登出/離線事件，4-聊天室聊天消息，5-音視頻時長、白板時長消息，6-音視頻白板大小、下載地址消息，7-單聊消息撤回，8-羣聊消息撤回，9-彙報主播、管理員進出聊天室事件消息，10-彙報專線電話通話結束回調抄送的消息，11-彙報短信回執抄送的消息，12-彙報短信上行消息，13-彙報用戶進出音視頻/白板房間的消息，14-彙報聊天室隊列操作的事件消息，20-易盾異步反垃圾結果信息',
`remind_type` tinyint(4) DEFAULT '0' COMMENT '提醒類型：0-普通消息，1-客服進入，2-客戶進入，3-客服進入歡迎提醒，4-敏感詞命中提醒消息',
`conv_type` varchar(20) DEFAULT '' COMMENT '場景類型：TEAM',
`scene` tinyint(4) DEFAULT '0' COMMENT '場景：0-team，1-p2p，2-superTeam',
`text` varchar(5000) DEFAULT '' COMMENT '內容',
`attach` varchar(5000) DEFAULT '' COMMENT '附加消息',
`team_attach_type` tinyint(4) DEFAULT NULL COMMENT '羣通知類型：默認null（普通消息），0-更新羣，1-拉人入羣，2-踢人出羣，3-接受入羣邀請，4-通過入羣邀請，5-添加羣管理員，6-移除羣管理員，7-主動退羣，8-解散羣，9-轉讓羣，10-更新羣成員禁言狀態',
`send_type` tinyint(4) DEFAULT '0' COMMENT '發送人類型：0-用戶，1-客服，2-藥師，3-醫生',
`send_client_ip` varchar(20) DEFAULT '' COMMENT '發送端ip',
`send_client_port` varchar(5) DEFAULT '' COMMENT '發送端端口',
`send_client_type` varchar(10) DEFAULT '' COMMENT '發送客戶端類型： AOS、IOS、PC、WINPHONE、WEB、REST',
`send_device_id` varchar(32) DEFAULT '' COMMENT '發送端設備編號',
`send_nick` varchar(50) DEFAULT '' COMMENT '發送人暱稱',
`send_id` varchar(32) DEFAULT '' COMMENT '發送人id（對應客服表和用戶表的accid）',
`send_at` bigint(11) DEFAULT NULL COMMENT '發送時間',
`msg_receipt_time` bigint(11) DEFAULT NULL COMMENT '已讀回執時間戳，如果有此字段, 說明此時間戳之前的所有消息對方均已讀',
`is_revoke` tinyint(4) DEFAULT '0' COMMENT '是否撤銷：0-否，1-是',
`revoke_at` bigint(11) DEFAULT '0' COMMENT '撤銷時間',
`custom_apns_text` varchar(200) DEFAULT '' COMMENT '自定義系統通知消息推送文本。僅在convType爲CUSTOM_PERSON或CUSTOM_TEAM時含此字段',
`ext` varchar(64) DEFAULT '' COMMENT '消息擴展字段',
`antispam` varchar(12) DEFAULT '' COMMENT '標識是否被反垃圾(‘’，true，false)',
`yidun_res` varchar(500) DEFAULT '' COMMENT '易盾反垃圾的原始處理細節',
`msg_status` tinyint(4) DEFAULT '0' COMMENT '消息發送狀態：0-發送成功，1-發送中，2-發送失敗',
`status` tinyint(4) DEFAULT '0' COMMENT '狀態：有效 0：無效',
PRIMARY KEY (`id`),
UNIQUE KEY `index_tt` (`id`,`team_msg_id`,`team_client_msg_id`) USING BTREE,
KEY `index_team_id_on_im_team_message` (`team_id`),
KEY `index_send_at_on_im_team_message` (`send_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='IM消息表';
SET FOREIGN_KEY_CHECKS = 1;
配置分庫分表與讀寫分離
根據羣id進行分庫，根據羣消息id進行分表，詳細配置見下：
application.properties
server.port=8888
spring.main.allow-bean-definition-overriding=true
#---------------------------------
# mybatis
#---------------------------------
mybatis.type-aliases-package=com.zzx.sharding.entity
#mybatis.type-handlers-package=com.example.typehandler
mybatis.configuration.map-underscore-to-camel-case=true
mybatis.configuration.default-fetch-size=100
mybatis.configuration.default-statement-timeout=3000
mybatis.mapper-locations=classpath*:mapper/*.xml
#--------------------------------
# ShardingSphere
#---------------------------------
spring.shardingsphere.datasource.names=master0,slave0,master1,slave1
spring.shardingsphere.datasource.master0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.master0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.master0.url=jdbc:mysql://localhost:3342/im_bis?useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true
spring.shardingsphere.datasource.master0.username=root
spring.shardingsphere.datasource.master0.password=123456
spring.shardingsphere.datasource.slave0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.slave0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.slave0.url=jdbc:mysql://localhost:3343/im_bis?useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true
spring.shardingsphere.datasource.slave0.username=root
spring.shardingsphere.datasource.slave0.password=123456
spring.shardingsphere.datasource.master1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.master1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.master1.url=jdbc:mysql://localhost:3344/im_bis?useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true
spring.shardingsphere.datasource.master1.username=root
spring.shardingsphere.datasource.master1.password=123456
spring.shardingsphere.datasource.slave1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.slave1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.slave1.url=jdbc:mysql://localhost:3345/im_bis?useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true
spring.shardingsphere.datasource.slave1.username=root
spring.shardingsphere.datasource.slave1.password=123456
Master0Prop
package com.zzx.sharding.config;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
/**
* @author zhouzhixiang
* @Date 2020-03-15
*/
@ConfigurationProperties(prefix = "spring.shardingsphere.datasource.master0")
@Data
public class Master0Prop {
private String url;
private String username;
private String password;
private String type;
private String driverClassName;
}
Master1Prop
package com.zzx.sharding.config;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
/**
* @author zhouzhixiang
* @Date 2020-03-15
*/
@ConfigurationProperties(prefix = "spring.shardingsphere.datasource.master1")
@Data
public class Master1Prop {
private String url;
private String username;
private String password;
private String type;
private String driverClassName;
}
Slave0Prop
package com.zzx.sharding.config;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
/**
* @author zhouzhixiang
* @Date 2020-03-15
*/
@ConfigurationProperties(prefix = "spring.shardingsphere.datasource.slave0")
@Data
public class Slave0Prop {
private String url;
private String username;
private String password;
private String type;
private String driverClassName;
}
Slave1Prop
package com.zzx.sharding.config;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
/**
* @author zhouzhixiang
* @Date 2020-03-15
*/
@ConfigurationProperties(prefix = "spring.shardingsphere.datasource.slave1")
@Data
public class Slave1Prop {
private String url;
private String username;
private String password;
private String type;
private String driverClassName;
}
DataSourceConfig
package com.zzx.sharding.config;
import com.alibaba.druid.pool.DruidDataSource;
import com.zzx.sharding.utils.DataSourceUtil;
import lombok.extern.slf4j.Slf4j;
import org.apache.ibatis.session.SqlSessionFactory;
import org.apache.shardingsphere.api.config.masterslave.LoadBalanceStrategyConfiguration;
import org.apache.shardingsphere.api.config.masterslave.MasterSlaveRuleConfiguration;
import org.apache.shardingsphere.api.config.sharding.KeyGeneratorConfiguration;
import org.apache.shardingsphere.api.config.sharding.ShardingRuleConfiguration;
import org.apache.shardingsphere.api.config.sharding.TableRuleConfiguration;
import org.apache.shardingsphere.api.config.sharding.strategy.InlineShardingStrategyConfiguration;
import org.apache.shardingsphere.shardingjdbc.api.ShardingDataSourceFactory;
import org.mybatis.spring.SqlSessionFactoryBean;
import org.mybatis.spring.SqlSessionTemplate;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import javax.sql.DataSource;
import java.sql.SQLException;
import java.util.*;
/**
* @author zhouzhixiang
*/
@Configuration
@EnableTransactionManagement(proxyTargetClass = true)
@EnableConfigurationProperties({Master0Prop.class, Master1Prop.class, Slave0Prop.class, Slave1Prop.class})
@Slf4j
@MapperScan(basePackages = "com.zzx.sharding.mapper", sqlSessionTemplateRef = "sqlSessionTemplate")
public class DataSourceConfig {
/**
* 配置數據源0，數據源的名稱最好要有一定的規則，方便配置分庫的計算規則
*
* @return
*/
@Bean(name = "master0")
public DataSource master0(Master0Prop masterProp) {
Map<String, Object> dsMap = new HashMap<>();
dsMap.put("type", masterProp.getType());
dsMap.put("url", masterProp.getUrl());
dsMap.put("username", masterProp.getUsername());
dsMap.put("password", masterProp.getPassword());
dsMap.put("driverClassName", masterProp.getDriverClassName());
DruidDataSource ds = (DruidDataSource) DataSourceUtil.buildDataSource(dsMap);
// 每個分區最大的連接數
ds.setMaxActive(20);
// 每個分區最小的連接數
ds.setMinIdle(5);
return ds;
}
/**
* 配置數據源0，數據源的名稱最好要有一定的規則，方便配置分庫的計算規則
*
* @return
*/
@Bean(name = "master1")
public DataSource master1(Master1Prop masterProp) {
Map<String, Object> dsMap = new HashMap<>();
dsMap.put("type", masterProp.getType());
dsMap.put("url", masterProp.getUrl());
dsMap.put("username", masterProp.getUsername());
dsMap.put("password", masterProp.getPassword());
dsMap.put("driverClassName", masterProp.getDriverClassName());
DruidDataSource ds = (DruidDataSource) DataSourceUtil.buildDataSource(dsMap);
// 每個分區最大的連接數
ds.setMaxActive(20);
// 每個分區最小的連接數
ds.setMinIdle(5);
return ds;
}
/**
* 配置數據源0，數據源的名稱最好要有一定的規則，方便配置分庫的計算規則
*
* @return
*/
@Bean(name = "slave0")
public DataSource slave0(Slave0Prop slave0Prop) {
Map<String, Object> dsMap = new HashMap<>();
dsMap.put("type", slave0Prop.getType());
dsMap.put("url", slave0Prop.getUrl());
dsMap.put("username", slave0Prop.getUsername());
dsMap.put("password", slave0Prop.getPassword());
dsMap.put("driverClassName", slave0Prop.getDriverClassName());
DruidDataSource ds = (DruidDataSource) DataSourceUtil.buildDataSource(dsMap);
// 每個分區最大的連接數
ds.setMaxActive(20);
// 每個分區最小的連接數
ds.setMinIdle(5);
return ds;
}
/**
* 配置數據源0，數據源的名稱最好要有一定的規則，方便配置分庫的計算規則
*
* @return
*/
@Bean(name = "slave1")
public DataSource slave1(Slave1Prop slave1Prop) {
Map<String, Object> dsMap = new HashMap<>();
dsMap.put("type", slave1Prop.getType());
dsMap.put("url", slave1Prop.getUrl());
dsMap.put("username", slave1Prop.getUsername());
dsMap.put("password", slave1Prop.getPassword());
dsMap.put("driverClassName", slave1Prop.getDriverClassName());
DruidDataSource ds = (DruidDataSource) DataSourceUtil.buildDataSource(dsMap);
// 每個分區最大的連接數
ds.setMaxActive(20);
// 每個分區最小的連接數
ds.setMinIdle(5);
return ds;
}
@Bean("dataSource")
public DataSource dataSource(@Qualifier("master0") DataSource master0, @Qualifier("master1") DataSource master1, @Qualifier("slave0") DataSource slave0, @Qualifier("slave1") DataSource slave1) throws SQLException {
// 配置真實數據源
Map<String, DataSource> dataSourceMap = new HashMap<>();
dataSourceMap.put("master0", master0);
dataSourceMap.put("master1", master1);
dataSourceMap.put("slave0", slave0);
dataSourceMap.put("slave1", slave1);
List<String> slave0List = new ArrayList<>();
slave0List.add("slave0");
List<String> slave1List = new ArrayList<>();
slave1List.add("slave1");
// 主從策略
LoadBalanceStrategyConfiguration loadBalanceStrategyConfiguration = new LoadBalanceStrategyConfiguration("round_robin");
MasterSlaveRuleConfiguration master0SlaveRuleConfiguration = new MasterSlaveRuleConfiguration("master0", "master0", slave0List, loadBalanceStrategyConfiguration);
MasterSlaveRuleConfiguration master1SlaveRuleConfiguration = new MasterSlaveRuleConfiguration("master1", "master1", slave1List, loadBalanceStrategyConfiguration);
// 打開shardingsphere sql日誌
Properties properties = new Properties();
properties.setProperty("sql.show", Boolean.TRUE.toString());
// 配置分片規則 分庫分表 讀寫分離
ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
shardingRuleConfig.getMasterSlaveRuleConfigs().add(master0SlaveRuleConfiguration);
shardingRuleConfig.getMasterSlaveRuleConfigs().add(master1SlaveRuleConfiguration);
// 配置消息表分庫分表
shardingRuleConfig.getTableRuleConfigs().add(getImTeamMessageRuleConfiguration());
// 獲取數據源對象
DataSource dataSource = ShardingDataSourceFactory.createDataSource(dataSourceMap, shardingRuleConfig, properties);
return dataSource;
}
/** 消息表——im_team_message */
TableRuleConfiguration getImTeamMessageRuleConfiguration() {
TableRuleConfiguration result = new TableRuleConfiguration("im_team_message", "master$->{0..1}.im_team_message$->{0..2}");
result.setDatabaseShardingStrategyConfig(new InlineShardingStrategyConfiguration("team_id", "master$->{team_id % 2}"));
result.setTableShardingStrategyConfig(new InlineShardingStrategyConfiguration("team_msg_id", "im_team_message$->{team_msg_id % 3}"));
result.setKeyGeneratorConfig(new KeyGeneratorConfiguration("SNOWFLAKE", "id"));
return result;
}
/** 消息表——im_team_message */
@Bean
public DataSourceTransactionManager shardTransactionManager(@Qualifier("dataSource") DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
@Bean("sqlSessionFactory")
@Primary
public SqlSessionFactory sqlSessionFactory(@Qualifier("dataSource") DataSource dataSource) throws Exception {
SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
bean.setDataSource(dataSource);
bean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath*:mapper/*.xml"));
return bean.getObject();
}
@Bean("sqlSessionTemplate")
@Primary
public SqlSessionTemplate sqlSessionTemplate(@Qualifier("sqlSessionFactory") SqlSessionFactory sqlSessionFactory) throws Exception {
return new SqlSessionTemplate(sqlSessionFactory);
}
}
驗證
添加數據——寫入主庫0或主庫1
postman執行： localhost:8888/teamMessage/batchAdd
代碼
@PostMapping("batchAdd")
public ServiceResponse batchAdd(HttpServletRequest request, HttpServletResponse response) throws IOException {
ServiceResponse result = new ServiceResponse<>();
for (int i = 0; i < 20; i++) {
TeamMessage vo = new TeamMessage();
vo.setSendAt(new Date().getTime());
vo.setTeamId(1000);
vo.setTeamClientMsgId(UUID.randomUUID().toString());
vo.setTeamMsgId(SnowIdUtils.uniqueLong());
vo.setId(SnowIdUtils.uniqueLong());
vo.setText("消息內容-"+UUID.randomUUID());
this.teamMessageServiceApi.insert(vo);
}
result.setCode(ResponseEnum.SUCCESS.getCode());
result.setMsg(ResponseEnum.SUCCESS.getName());
return result;
}
羣id爲1000時，路由到主庫0，寫入主庫0成功
日誌：
2020-03-11 18:25:06.525 INFO 46360 --- [nio-8888-exec-8] ShardingSphere-SQL : Actual SQL: master0 ::: INSERT IGNORE INTO ....
主庫0：表中的數據根據羣消息id路由到各表中：
主庫1: 表中沒有數據
![[外鏈圖片轉存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-uaEwBAF5-1584290836592)(evernotecid://6438F78B-FBEF-42D1-A152-F4935CA4BBB8/appyinxiangcom/15654258/ENResource/p11669)]](https://pic1.xuehuaimg.com/proxy/csdn/https://img-blog.csdnimg.cn/20200316005038935.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RhbmllbHpob3U4ODg=,size_16,color_FFFFFF,t_70)
羣id爲1001時，路由到主庫1，寫入主庫1成功
日誌：
2020-03-11 18:30:12.345 INFO 46360 --- [nio-8888-exec-8] ShardingSphere-SQL : Actual SQL: master1 ::: INSERT IGNORE INTO ....
主庫1：表中的數據根據羣消息id路由到各表中：
主庫0: 表中沒有新增數據
。。。
分庫驗證成功！
查詢數據——從庫查詢
查詢所有數據
由於分庫，且每個主庫僅一個從庫，我們執行查詢所有數據操作，shardingsphere會幫我們從各從庫各分表中採用聚合的方式獲取數據。後面會根據shardingsphere原理單獨說明如何對聚合的數據進行操作。
控制檯sql日誌：
2020-03-11 18:35:29.399 INFO 46360 --- [nio-8888-exec-1] ShardingSphere-SQL : Rule Type: sharding
2020-03-11 18:35:29.399 INFO 46360 --- [nio-8888-exec-1] ShardingSphere-SQL : Logic SQL: SELECT id,。。。。。 FROM im_team_message WHERE STATUS = 0
2020-03-11 18:35:29.399 INFO 46360 --- [nio-8888-exec-1] ShardingSphere-SQL : SQLStatement: SelectStatement(super=DQLStatement(super=AbstractSQLStatement(type=DQL, tables=Tables(tables=[Table(name=im_team_message, alias=Optional.absent())]), routeConditions=Conditions(orCondition=OrCondition(andConditions=[])), encryptConditions=Conditions(orCondition=OrCondition(andConditions=[])), sqlTokens=[TableToken(tableName=im_team_message, quoteCharacter=NONE, schemaNameLength=0)], parametersIndex=0, Logic SQL: SELECT id,。。。。。 FROM im_team_message WHERE status = 0)), containStar=false, firstSelectItemStartIndex=14, selectListStopIndex=410, groupByLastIndex=0, items=[CommonSelectItem(expression=id, alias=Optional.absent()), CommonSelectItem(expression=team_msg_id, alias=Optional.absent()), CommonSelectItem(expression=team_client_msg_id, alias=Optional.absent()), CommonSelectItem(expression=msg_type, alias=Optional.absent()), CommonSelectItem(expression=remind_type, alias=Optional.absent()), CommonSelectItem(expression=conv_type, alias=Optional.absent()), CommonSelectItem(expression=scene, alias=Optional.absent()), CommonSelectItem(expression=text, alias=Optional.absent()), CommonSelectItem(expression=team_attach_type, alias=Optional.absent()), CommonSelectItem(expression=send_type, alias=Optional.absent()), CommonSelectItem(expression=send_client_ip, alias=Optional.absent()), CommonSelectItem(expression=send_client_port, alias=Optional.absent()), CommonSelectItem(expression=send_client_type, alias=Optional.absent()), CommonSelectItem(expression=send_device_id, alias=Optional.absent()), CommonSelectItem(expression=send_nick, alias=Optional.absent()), CommonSelectItem(expression=team_id, alias=Optional.absent()), CommonSelectItem(expression=send_id, alias=Optional.absent()), CommonSelectItem(expression=send_at, alias=Optional.absent()), CommonSelectItem(expression=msg_receipt_time, alias=Optional.absent()), CommonSelectItem(expression=is_revoke, alias=Optional.absent()), CommonSelectItem(expression=revoke_at, alias=Optional.absent()), CommonSelectItem(expression=msg_status, alias=Optional.absent()), CommonSelectItem(expression=event_type, alias=Optional.absent()), CommonSelectItem(expression=status, alias=Optional.absent()), CommonSelectItem(expression=attach, alias=Optional.absent()), CommonSelectItem(expression=custom_apns_text, alias=Optional.absent()), CommonSelectItem(expression=ext, alias=Optional.absent()), CommonSelectItem(expression=antispam, alias=Optional.absent()), CommonSelectItem(expression=yidun_res, alias=Optional.absent())], groupByItems=[], orderByItems=[], limit=null, subqueryStatement=null, subqueryStatements=[], subqueryConditions=[])
2020-03-11 18:35:29.399 INFO 46360 --- [nio-8888-exec-1] ShardingSphere-SQL : Actual SQL: slave0 ::: select 。。。。
。。。
2020-03-11 18:35:29.399 INFO 46360 --- [nio-8888-exec-1] ShardingSphere-SQL : Actual SQL: slave1 ::: select
。。。
部分日誌省略，從上面可以看得出，查詢所有數據時候，shardingsphere從從庫0和從庫1各表中將數據進行聚合處理返回給調用接口。
返回數據
![[外鏈圖片轉存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-2ZZ2hf0D-1584290836594)(evernotecid://6438F78B-FBEF-42D1-A152-F4935CA4BBB8/appyinxiangcom/15654258/ENResource/p11675)]](https://pic1.xuehuaimg.com/proxy/csdn/https://img-blog.csdnimg.cn/20200316005153480.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2RhbmllbHpob3U4ODg=,size_16,color_FFFFFF,t_70)
由於執行了兩次兩個羣組的新增數據，總共40條數據
分庫分表驗證成功！
問題記錄
springboot2.1.5默認使用mysql-connECTOR-java8.x的版本，這個版本的驅動使用shardingsphere時會有bug，推薦使用mysql-connector-java5.1.47版本
