Java猿社區—ShardingSphere4.0.0-RC1實現分庫分表+讀寫分離
文章目錄
參考:
基於Docker的Mysql主從複製搭建
ShardingSphere官網
技術體系
Springboot2.1.5 + shardingsphere4.0.0-RC1 + Mysql5.7 + mysql-connector-java5.1.47
背景
目前公司正在進行的在線客服IM項目,考慮未來會話消息請求流量劇增以及消息存儲帶來的挑戰,單一數據庫無法支撐,需要分庫&分表、讀寫分離以應對高併發帶來的挑戰。
基於市面上較爲流行的幾個數據庫中間件我們結合公司技術體系,再三考慮,決定使用ShardingSphere作爲在線客服項目的數據庫中間件。
ShardingSphere介紹
Apache ShardingSphere(Incubator) 是一套開源的分佈式數據庫中間件解決方案組成的生態圈,它由Sharding-JDBC、Sharding-Proxy和Sharding-Sidecar(規劃中)這3款相互獨立,卻又能夠混合部署配合使用的產品組成。它們均提供標準化的數據分片、分佈式事務和數據庫治理功能,可適用於如Java同構、異構語言、雲原生等各種多樣化的應用場景。(來自官網)
ShardingShpere支持的功能
數據分片
- 分庫、分表
- 讀寫分離
- 分片策略定製化
- 無中心化分佈式主鍵
分佈式事務
- 標準化事務接口
- XA強一致事務
- 柔性事務
技術準備
mysql安裝配置
具體安裝配置請參見網絡博客:如基於Docker的Mysql主從複製搭建
POM配置
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.zzx.shardingsphere</groupId>
<artifactId>zzx-shardingsphere</artifactId>
<packaging>pom</packaging>
<version>1.0-SNAPSHOT</version>
<modules>
<module>db-read-write</module>
<module>db-table-read-write</module>
</modules>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.5.RELEASE</version>
<relativePath/>
</parent>
<properties>
<java.version>1.8</java.version>
<mybatis-spring-boot>2.0.1</mybatis-spring-boot>
<druid>1.1.16</druid>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<shardingsphere.version>4.0.0-RC1</shardingsphere.version>
<fastjson.version>1.2.38</fastjson.version>
<mysql-connector-java.version>5.1.47</mysql-connector-java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>${mybatis-spring-boot}</version>
</dependency>
<!--mybatis驅動-->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${mysql-connector-java.version}</version>
</dependency>
<!--druid數據源-->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid-spring-boot-starter</artifactId>
<version>${druid}</version>
</dependency>
<!--shardingsphere版本-->
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-namespace</artifactId>
<version>${shardingsphere.version}</version>
</dependency>
<!--lombok實體工具-->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>${fastjson.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
分庫分表+讀寫分離
mysql配置環境
主庫0: localhost:3342/im_bis
主庫0->從庫0: localhost:3343/im_bis
主庫1: localhost:3344/im_bis
主庫1->從庫1: localhost:3345/im_bis
配置從庫0和從庫1分別作爲主庫0和主庫1的從庫,具體配置參考上面提到的基於Docker搭建主從複製網站
sql腳本
/*
Navicat Premium Data Transfer
Source Server : 192.168.89.53
Source Server Type : MySQL
Source Server Version : 50726
Source Host : 192.168.89.53:3306
Source Schema : ddky_im_bis
Target Server Type : MySQL
Target Server Version : 50726
File Encoding : 65001
Date: 08/03/2020 13:18:39
*/
DROP SCHEMA IF EXISTS im_bis;
CREATE SCHEMA IF NOT EXISTS im_bis;
use im_bis
SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for im_team_message
-- ----------------------------
DROP TABLE IF EXISTS `im_team_message0`;
CREATE TABLE `im_team_message0` (
`id` bigint(20) NOT NULL COMMENT '主鍵id',
`team_msg_id` bigint(20) DEFAULT NULL COMMENT '羣消息id(服務端生成消息id)',
`team_client_msg_id` varchar(32) DEFAULT '' COMMENT '羣消息id(客戶端生成)',
`team_id` bigint(20) DEFAULT NULL COMMENT '羣組id(對應羣組的accid)',
`msg_type` varchar(15) DEFAULT 'TEXT' COMMENT '消息類型:TEXT-文本,PICTURE-圖片,AUDIO-音頻,VIDEO-視頻,FILE-文件,GEO-地理位置,CUSTOM-自定義,TIP-提醒,ROBOT-AI機器人,NOTICATION-羣通知,TEAM_INVITE-邀請入羣,TEAM_INVITE_REJECT-拒絕邀請,CUSTOM_TEAM_MSG -羣組自定義系統通知',
`event_type` tinyint(4) DEFAULT '1' COMMENT '事件類型:1-會話類型消息(p2p消息、羣聊消息、自定義系統通知、雲信內置系統通知),2-登錄事件,3-登出/離線事件,4-聊天室聊天消息,5-音視頻時長、白板時長消息,6-音視頻白板大小、下載地址消息,7-單聊消息撤回,8-羣聊消息撤回,9-彙報主播、管理員進出聊天室事件消息,10-彙報專線電話通話結束回調抄送的消息,11-彙報短信回執抄送的消息,12-彙報短信上行消息,13-彙報用戶進出音視頻/白板房間的消息,14-彙報聊天室隊列操作的事件消息,20-易盾異步反垃圾結果信息',
`remind_type` tinyint(4) DEFAULT '0' COMMENT '提醒類型:0-普通消息,1-客服進入,2-客戶進入,3-客服進入歡迎提醒,4-敏感詞命中提醒消息',
`conv_type` varchar(20) DEFAULT '' COMMENT '場景類型:TEAM',
`scene` tinyint(4) DEFAULT '0' COMMENT '場景:0-team,1-p2p,2-superTeam',
`text` varchar(5000) DEFAULT '' COMMENT '內容',
`attach` varchar(5000) DEFAULT '' COMMENT '附加消息',
`team_attach_type` tinyint(4) DEFAULT NULL COMMENT '羣通知類型:默認null(普通消息),0-更新羣,1-拉人入羣,2-踢人出羣,3-接受入羣邀請,4-通過入羣邀請,5-添加羣管理員,6-移除羣管理員,7-主動退羣,8-解散羣,9-轉讓羣,10-更新羣成員禁言狀態',
`send_type` tinyint(4) DEFAULT '0' COMMENT '發送人類型:0-用戶,1-客服,2-藥師,3-醫生',
`send_client_ip` varchar(20) DEFAULT '' COMMENT '發送端ip',
`send_client_port` varchar(5) DEFAULT '' COMMENT '發送端端口',
`send_client_type` varchar(10) DEFAULT '' COMMENT '發送客戶端類型: AOS、IOS、PC、WINPHONE、WEB、REST',
`send_device_id` varchar(32) DEFAULT '' COMMENT '發送端設備編號',
`send_nick` varchar(50) DEFAULT '' COMMENT '發送人暱稱',
`send_id` varchar(32) DEFAULT '' COMMENT '發送人id(對應客服表和用戶表的accid)',
`send_at` bigint(11) DEFAULT NULL COMMENT '發送時間',
`msg_receipt_time` bigint(11) DEFAULT NULL COMMENT '已讀回執時間戳,如果有此字段, 說明此時間戳之前的所有消息對方均已讀',
`is_revoke` tinyint(4) DEFAULT '0' COMMENT '是否撤銷:0-否,1-是',
`revoke_at` bigint(11) DEFAULT '0' COMMENT '撤銷時間',
`custom_apns_text` varchar(200) DEFAULT '' COMMENT '自定義系統通知消息推送文本。僅在convType爲CUSTOM_PERSON或CUSTOM_TEAM時含此字段',
`ext` varchar(64) DEFAULT '' COMMENT '消息擴展字段',
`antispam` varchar(12) DEFAULT '' COMMENT '標識是否被反垃圾(‘’,true,false)',
`yidun_res` varchar(500) DEFAULT '' COMMENT '易盾反垃圾的原始處理細節',
`msg_status` tinyint(4) DEFAULT '0' COMMENT '消息發送狀態:0-發送成功,1-發送中,2-發送失敗',
`status` tinyint(4) DEFAULT '0' COMMENT '狀態:有效 0:無效',
PRIMARY KEY (`id`),
UNIQUE KEY `index_tt` (`id`,`team_msg_id`,`team_client_msg_id`) USING BTREE,
KEY `index_team_id_on_im_team_message` (`team_id`),
KEY `index_send_at_on_im_team_message` (`send_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='IM消息表';
SET FOREIGN_KEY_CHECKS = 1;
/*
Navicat Premium Data Transfer
Source Server : 192.168.89.53
Source Server Type : MySQL
Source Server Version : 50726
Source Host : 192.168.89.53:3306
Source Schema : ddky_im_bis
Target Server Type : MySQL
Target Server Version : 50726
File Encoding : 65001
Date: 08/03/2020 13:18:39
*/
SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for im_team_message
-- ----------------------------
DROP TABLE IF EXISTS `im_team_message1`;
CREATE TABLE `im_team_message1` (
`id` bigint(20) NOT NULL COMMENT '主鍵id',
`team_msg_id` bigint(20) DEFAULT NULL COMMENT '羣消息id(服務端生成消息id)',
`team_client_msg_id` varchar(32) DEFAULT '' COMMENT '羣消息id(客戶端生成)',
`team_id` bigint(20) DEFAULT NULL COMMENT '羣組id(對應羣組的accid)',
`msg_type` varchar(15) DEFAULT 'TEXT' COMMENT '消息類型:TEXT-文本,PICTURE-圖片,AUDIO-音頻,VIDEO-視頻,FILE-文件,GEO-地理位置,CUSTOM-自定義,TIP-提醒,ROBOT-AI機器人,NOTICATION-羣通知,TEAM_INVITE-邀請入羣,TEAM_INVITE_REJECT-拒絕邀請,CUSTOM_TEAM_MSG -羣組自定義系統通知',
`event_type` tinyint(4) DEFAULT '1' COMMENT '事件類型:1-會話類型消息(p2p消息、羣聊消息、自定義系統通知、雲信內置系統通知),2-登錄事件,3-登出/離線事件,4-聊天室聊天消息,5-音視頻時長、白板時長消息,6-音視頻白板大小、下載地址消息,7-單聊消息撤回,8-羣聊消息撤回,9-彙報主播、管理員進出聊天室事件消息,10-彙報專線電話通話結束回調抄送的消息,11-彙報短信回執抄送的消息,12-彙報短信上行消息,13-彙報用戶進出音視頻/白板房間的消息,14-彙報聊天室隊列操作的事件消息,20-易盾異步反垃圾結果信息',
`remind_type` tinyint(4) DEFAULT '0' COMMENT '提醒類型:0-普通消息,1-客服進入,2-客戶進入,3-客服進入歡迎提醒,4-敏感詞命中提醒消息',
`conv_type` varchar(20) DEFAULT '' COMMENT '場景類型:TEAM',
`scene` tinyint(4) DEFAULT '0' COMMENT '場景:0-team,1-p2p,2-superTeam',
`text` varchar(5000) DEFAULT '' COMMENT '內容',
`attach` varchar(5000) DEFAULT '' COMMENT '附加消息',
`team_attach_type` tinyint(4) DEFAULT NULL COMMENT '羣通知類型:默認null(普通消息),0-更新羣,1-拉人入羣,2-踢人出羣,3-接受入羣邀請,4-通過入羣邀請,5-添加羣管理員,6-移除羣管理員,7-主動退羣,8-解散羣,9-轉讓羣,10-更新羣成員禁言狀態',
`send_type` tinyint(4) DEFAULT '0' COMMENT '發送人類型:0-用戶,1-客服,2-藥師,3-醫生',
`send_client_ip` varchar(20) DEFAULT '' COMMENT '發送端ip',
`send_client_port` varchar(5) DEFAULT '' COMMENT '發送端端口',
`send_client_type` varchar(10) DEFAULT '' COMMENT '發送客戶端類型: AOS、IOS、PC、WINPHONE、WEB、REST',
`send_device_id` varchar(32) DEFAULT '' COMMENT '發送端設備編號',
`send_nick` varchar(50) DEFAULT '' COMMENT '發送人暱稱',
`send_id` varchar(32) DEFAULT '' COMMENT '發送人id(對應客服表和用戶表的accid)',
`send_at` bigint(11) DEFAULT NULL COMMENT '發送時間',
`msg_receipt_time` bigint(11) DEFAULT NULL COMMENT '已讀回執時間戳,如果有此字段, 說明此時間戳之前的所有消息對方均已讀',
`is_revoke` tinyint(4) DEFAULT '0' COMMENT '是否撤銷:0-否,1-是',
`revoke_at` bigint(11) DEFAULT '0' COMMENT '撤銷時間',
`custom_apns_text` varchar(200) DEFAULT '' COMMENT '自定義系統通知消息推送文本。僅在convType爲CUSTOM_PERSON或CUSTOM_TEAM時含此字段',
`ext` varchar(64) DEFAULT '' COMMENT '消息擴展字段',
`antispam` varchar(12) DEFAULT '' COMMENT '標識是否被反垃圾(‘’,true,false)',
`yidun_res` varchar(500) DEFAULT '' COMMENT '易盾反垃圾的原始處理細節',
`msg_status` tinyint(4) DEFAULT '0' COMMENT '消息發送狀態:0-發送成功,1-發送中,2-發送失敗',
`status` tinyint(4) DEFAULT '0' COMMENT '狀態:有效 0:無效',
PRIMARY KEY (`id`),
UNIQUE KEY `index_tt` (`id`,`team_msg_id`,`team_client_msg_id`) USING BTREE,
KEY `index_team_id_on_im_team_message` (`team_id`),
KEY `index_send_at_on_im_team_message` (`send_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='IM消息表';
SET FOREIGN_KEY_CHECKS = 1;
/*
Navicat Premium Data Transfer
Source Server : 192.168.89.53
Source Server Type : MySQL
Source Server Version : 50726
Source Host : 192.168.89.53:3306
Source Schema : ddky_im_bis
Target Server Type : MySQL
Target Server Version : 50726
File Encoding : 65001
Date: 08/03/2020 13:18:39
*/
SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;
-- ----------------------------
-- Table structure for im_team_message
-- ----------------------------
DROP TABLE IF EXISTS `im_team_message2`;
CREATE TABLE `im_team_message2` (
`id` bigint(20) NOT NULL COMMENT '主鍵id',
`team_msg_id` bigint(20) DEFAULT NULL COMMENT '羣消息id(服務端生成消息id)',
`team_client_msg_id` varchar(32) DEFAULT '' COMMENT '羣消息id(客戶端生成)',
`team_id` bigint(20) DEFAULT NULL COMMENT '羣組id(對應羣組的accid)',
`msg_type` varchar(15) DEFAULT 'TEXT' COMMENT '消息類型:TEXT-文本,PICTURE-圖片,AUDIO-音頻,VIDEO-視頻,FILE-文件,GEO-地理位置,CUSTOM-自定義,TIP-提醒,ROBOT-AI機器人,NOTICATION-羣通知,TEAM_INVITE-邀請入羣,TEAM_INVITE_REJECT-拒絕邀請,CUSTOM_TEAM_MSG -羣組自定義系統通知',
`event_type` tinyint(4) DEFAULT '1' COMMENT '事件類型:1-會話類型消息(p2p消息、羣聊消息、自定義系統通知、雲信內置系統通知),2-登錄事件,3-登出/離線事件,4-聊天室聊天消息,5-音視頻時長、白板時長消息,6-音視頻白板大小、下載地址消息,7-單聊消息撤回,8-羣聊消息撤回,9-彙報主播、管理員進出聊天室事件消息,10-彙報專線電話通話結束回調抄送的消息,11-彙報短信回執抄送的消息,12-彙報短信上行消息,13-彙報用戶進出音視頻/白板房間的消息,14-彙報聊天室隊列操作的事件消息,20-易盾異步反垃圾結果信息',
`remind_type` tinyint(4) DEFAULT '0' COMMENT '提醒類型:0-普通消息,1-客服進入,2-客戶進入,3-客服進入歡迎提醒,4-敏感詞命中提醒消息',
`conv_type` varchar(20) DEFAULT '' COMMENT '場景類型:TEAM',
`scene` tinyint(4) DEFAULT '0' COMMENT '場景:0-team,1-p2p,2-superTeam',
`text` varchar(5000) DEFAULT '' COMMENT '內容',
`attach` varchar(5000) DEFAULT '' COMMENT '附加消息',
`team_attach_type` tinyint(4) DEFAULT NULL COMMENT '羣通知類型:默認null(普通消息),0-更新羣,1-拉人入羣,2-踢人出羣,3-接受入羣邀請,4-通過入羣邀請,5-添加羣管理員,6-移除羣管理員,7-主動退羣,8-解散羣,9-轉讓羣,10-更新羣成員禁言狀態',
`send_type` tinyint(4) DEFAULT '0' COMMENT '發送人類型:0-用戶,1-客服,2-藥師,3-醫生',
`send_client_ip` varchar(20) DEFAULT '' COMMENT '發送端ip',
`send_client_port` varchar(5) DEFAULT '' COMMENT '發送端端口',
`send_client_type` varchar(10) DEFAULT '' COMMENT '發送客戶端類型: AOS、IOS、PC、WINPHONE、WEB、REST',
`send_device_id` varchar(32) DEFAULT '' COMMENT '發送端設備編號',
`send_nick` varchar(50) DEFAULT '' COMMENT '發送人暱稱',
`send_id` varchar(32) DEFAULT '' COMMENT '發送人id(對應客服表和用戶表的accid)',
`send_at` bigint(11) DEFAULT NULL COMMENT '發送時間',
`msg_receipt_time` bigint(11) DEFAULT NULL COMMENT '已讀回執時間戳,如果有此字段, 說明此時間戳之前的所有消息對方均已讀',
`is_revoke` tinyint(4) DEFAULT '0' COMMENT '是否撤銷:0-否,1-是',
`revoke_at` bigint(11) DEFAULT '0' COMMENT '撤銷時間',
`custom_apns_text` varchar(200) DEFAULT '' COMMENT '自定義系統通知消息推送文本。僅在convType爲CUSTOM_PERSON或CUSTOM_TEAM時含此字段',
`ext` varchar(64) DEFAULT '' COMMENT '消息擴展字段',
`antispam` varchar(12) DEFAULT '' COMMENT '標識是否被反垃圾(‘’,true,false)',
`yidun_res` varchar(500) DEFAULT '' COMMENT '易盾反垃圾的原始處理細節',
`msg_status` tinyint(4) DEFAULT '0' COMMENT '消息發送狀態:0-發送成功,1-發送中,2-發送失敗',
`status` tinyint(4) DEFAULT '0' COMMENT '狀態:有效 0:無效',
PRIMARY KEY (`id`),
UNIQUE KEY `index_tt` (`id`,`team_msg_id`,`team_client_msg_id`) USING BTREE,
KEY `index_team_id_on_im_team_message` (`team_id`),
KEY `index_send_at_on_im_team_message` (`send_at`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='IM消息表';
SET FOREIGN_KEY_CHECKS = 1;
配置分庫分表與讀寫分離
根據羣id進行分庫,根據羣消息id進行分表,詳細配置見下:
server.port=8888
spring.main.allow-bean-definition-overriding=true
#---------------------------------
# mybatis
#---------------------------------
mybatis.type-aliases-package=com.zzx.sharding.entity
#mybatis.type-handlers-package=com.example.typehandler
mybatis.configuration.map-underscore-to-camel-case=true
mybatis.configuration.default-fetch-size=100
mybatis.configuration.default-statement-timeout=3000
mybatis.mapper-locations=classpath*:mapper/*.xml
#--------------------------------
# druid
#---------------------------------
spring.datasource.type=com.alibaba.druid.pool.DruidDataSource
spring.datasource.druid.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.druid.filters=stat,wall,log4j2
spring.datasource.druid.max-active=50
spring.datasource.druid.initial-size=2
spring.datasource.druid.max-wait=60000
spring.datasource.druid.min-idle=2
spring.datasource.druid.keep-alive=true
spring.datasource.druid.time-between-eviction-runs-millis=60000
spring.datasource.druid.min-evictable-idle-time-millis=300000
spring.datasource.druid.validation-query=select 'x'
spring.datasource.druid.test-while-idle=true
spring.datasource.druid.test-on-borrow=false
spring.datasource.druid.test-on-return=false
spring.datasource.druid.pool-prepared-statements=true
spring.datasource.druid.max-open-prepared-statements=20
spring.datasource.druid.connection-properties=druid.stat.mergeSql=true;druid.stat.slowSqlMillis=5000;
spring.datasource.druid.use-global-data-source-stat=true
spring.datasource.druid.web-stat-filter.enabled=true
spring.datasource.druid.web-stat-filter.url-pattern=/*
spring.datasource.druid.web-stat-filter.exclusions='*.gif,*.png,*.jpg,*.html,*.js,*.css,*.ico,/druid/*'
spring.datasource.druid.stat-view-servlet.enabled=true
spring.datasource.druid.stat-view-servlet.url-pattern=/druid/*
spring.datasource.druid.stat-view-servlet.reset-enable=true
spring.datasource.druid.stat-view-servlet.login-username=admin
spring.datasource.druid.stat-view-servlet.login-password=admin
# 配置日誌輸出
spring.datasource.druid.filter.log4j2.enabled=true
spring.datasource.druid.filter.log4j2.statement-create-after-log-enabled=false
spring.datasource.druid.filter.log4j2.statement-close-after-log-enabled=false
spring.datasource.druid.filter.log4j2.result-set-open-after-log-enabled=false
spring.datasource.druid.filter.log4j2.result-set-close-after-log-enabled=false
#--------------------------------
# ShardingSphere
#---------------------------------
#打印sql
spring.shardingsphere.props.sql.show=true
spring.shardingsphere.datasource.names=master0,slave0,master1,slave1
spring.shardingsphere.datasource.master0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.master0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.master0.url=jdbc:mysql://localhost:3342/im_bis?useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true
spring.shardingsphere.datasource.master0.username=root
spring.shardingsphere.datasource.master0.password=123456
spring.shardingsphere.datasource.slave0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.slave0.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.slave0.url=jdbc:mysql://localhost:3343/im_bis?useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true
spring.shardingsphere.datasource.slave0.username=root
spring.shardingsphere.datasource.slave0.password=123456
spring.shardingsphere.datasource.master1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.master1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.master1.url=jdbc:mysql://localhost:3344/im_bis?useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true
spring.shardingsphere.datasource.master1.username=root
spring.shardingsphere.datasource.master1.password=123456
spring.shardingsphere.datasource.slave1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.slave1.driver-class-name=com.mysql.jdbc.Driver
spring.shardingsphere.datasource.slave1.url=jdbc:mysql://localhost:3345/im_bis?useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&useSSL=false&serverTimezone=GMT%2B8&autoReconnect=true&rewriteBatchedStatements=true
spring.shardingsphere.datasource.slave1.username=root
spring.shardingsphere.datasource.slave1.password=123456
#根據羣id分庫
spring.shardingsphere.sharding.default-database-strategy.inline.sharding-column=team_id
spring.shardingsphere.sharding.default-database-strategy.inline.algorithm-expression=master$->{team_id % 2}
#根據羣消息id分表
spring.shardingsphere.sharding.tables.im_team_message.actual-data-nodes=master$->{0..1}.im_team_message$->{0..2}
spring.shardingsphere.sharding.tables.im_team_message.table-strategy.inline.sharding-column=team_msg_id
spring.shardingsphere.sharding.tables.im_team_message.table-strategy.inline.algorithm-expression=im_team_message$->{team_msg_id % 3}
# 廢棄
#spring.shardingsphere.sharding.tables.im_team_message.key-generator.column=id
#spring.shardingsphere.sharding.tables.im_team_message.key-generator.type=SNOWFLAKE
#spring.shardingsphere.sharding.tables.im_team_message.key-generator.props.worker.id=123
#指定master0爲主庫,slave0爲它的從庫
spring.shardingsphere.sharding.master-slave-rules.master0.master-data-source-name=master0
spring.shardingsphere.sharding.master-slave-rules.master0.slave-data-source-names=slave0
#指定master1爲主庫,slave1爲它的從庫
spring.shardingsphere.sharding.master-slave-rules.master1.master-data-source-name=master1
spring.shardingsphere.sharding.master-slave-rules.master1.slave-data-source-names=slave1
驗證
添加數據——寫入主庫0或主庫1
postman執行: localhost:8888/teamMessage/batchAdd
代碼
@PostMapping("batchAdd")
public ServiceResponse batchAdd(HttpServletRequest request, HttpServletResponse response) throws IOException {
ServiceResponse result = new ServiceResponse<>();
for (int i = 0; i < 20; i++) {
TeamMessage vo = new TeamMessage();
vo.setSendAt(new Date().getTime());
vo.setTeamId(1000);
vo.setTeamClientMsgId(UUID.randomUUID().toString());
vo.setTeamMsgId(SnowIdUtils.uniqueLong());
vo.setId(SnowIdUtils.uniqueLong());
vo.setText("消息內容-"+UUID.randomUUID());
this.teamMessageServiceApi.insert(vo);
}
result.setCode(ResponseEnum.SUCCESS.getCode());
result.setMsg(ResponseEnum.SUCCESS.getName());
return result;
}
羣id爲1000時,路由到主庫0,寫入主庫0成功
日誌:
2020-03-11 18:25:06.525 INFO 46360 --- [nio-8888-exec-8] ShardingSphere-SQL : Actual SQL: master0 ::: INSERT IGNORE INTO ....
主庫0:表中的數據根據羣消息id路由到各表中:
主庫1: 表中沒有數據
羣id爲1001時,路由到主庫1,寫入主庫1成功
日誌:
2020-03-11 18:30:12.345 INFO 46360 --- [nio-8888-exec-8] ShardingSphere-SQL : Actual SQL: master1 ::: INSERT IGNORE INTO ....
主庫1:表中的數據根據羣消息id路由到各表中:
主庫0: 表中沒有新增數據
分庫驗證成功!
查詢數據——從庫查詢
查詢所有數據
由於分庫,且每個主庫僅一個從庫,我們執行查詢所有數據操作,shardingsphere會幫我們從各從庫各分表中採用聚合的方式獲取數據。後面會根據shardingsphere原理單獨說明如何對聚合的數據進行操作。
控制檯sql日誌:
2020-03-11 18:35:29.399 INFO 46360 --- [nio-8888-exec-1] ShardingSphere-SQL : Rule Type: sharding
2020-03-11 18:35:29.399 INFO 46360 --- [nio-8888-exec-1] ShardingSphere-SQL : Logic SQL: SELECT id,。。。。。 FROM im_team_message WHERE STATUS = 0
2020-03-11 18:35:29.399 INFO 46360 --- [nio-8888-exec-1] ShardingSphere-SQL : SQLStatement: SelectStatement(super=DQLStatement(super=AbstractSQLStatement(type=DQL, tables=Tables(tables=[Table(name=im_team_message, alias=Optional.absent())]), routeConditions=Conditions(orCondition=OrCondition(andConditions=[])), encryptConditions=Conditions(orCondition=OrCondition(andConditions=[])), sqlTokens=[TableToken(tableName=im_team_message, quoteCharacter=NONE, schemaNameLength=0)], parametersIndex=0, Logic SQL: SELECT id,。。。。。 FROM im_team_message WHERE status = 0)), containStar=false, firstSelectItemStartIndex=14, selectListStopIndex=410, groupByLastIndex=0, items=[CommonSelectItem(expression=id, alias=Optional.absent()), CommonSelectItem(expression=team_msg_id, alias=Optional.absent()), CommonSelectItem(expression=team_client_msg_id, alias=Optional.absent()), CommonSelectItem(expression=msg_type, alias=Optional.absent()), CommonSelectItem(expression=remind_type, alias=Optional.absent()), CommonSelectItem(expression=conv_type, alias=Optional.absent()), CommonSelectItem(expression=scene, alias=Optional.absent()), CommonSelectItem(expression=text, alias=Optional.absent()), CommonSelectItem(expression=team_attach_type, alias=Optional.absent()), CommonSelectItem(expression=send_type, alias=Optional.absent()), CommonSelectItem(expression=send_client_ip, alias=Optional.absent()), CommonSelectItem(expression=send_client_port, alias=Optional.absent()), CommonSelectItem(expression=send_client_type, alias=Optional.absent()), CommonSelectItem(expression=send_device_id, alias=Optional.absent()), CommonSelectItem(expression=send_nick, alias=Optional.absent()), CommonSelectItem(expression=team_id, alias=Optional.absent()), CommonSelectItem(expression=send_id, alias=Optional.absent()), CommonSelectItem(expression=send_at, alias=Optional.absent()), CommonSelectItem(expression=msg_receipt_time, alias=Optional.absent()), CommonSelectItem(expression=is_revoke, alias=Optional.absent()), CommonSelectItem(expression=revoke_at, alias=Optional.absent()), CommonSelectItem(expression=msg_status, alias=Optional.absent()), CommonSelectItem(expression=event_type, alias=Optional.absent()), CommonSelectItem(expression=status, alias=Optional.absent()), CommonSelectItem(expression=attach, alias=Optional.absent()), CommonSelectItem(expression=custom_apns_text, alias=Optional.absent()), CommonSelectItem(expression=ext, alias=Optional.absent()), CommonSelectItem(expression=antispam, alias=Optional.absent()), CommonSelectItem(expression=yidun_res, alias=Optional.absent())], groupByItems=[], orderByItems=[], limit=null, subqueryStatement=null, subqueryStatements=[], subqueryConditions=[])
2020-03-11 18:35:29.399 INFO 46360 --- [nio-8888-exec-1] ShardingSphere-SQL : Actual SQL: slave0 ::: select 。。。。
。。。
2020-03-11 18:35:29.399 INFO 46360 --- [nio-8888-exec-1] ShardingSphere-SQL : Actual SQL: slave1 ::: select
。。。
部分日誌省略,從上面可以看得出,查詢所有數據時候,shardingsphere從從庫0和從庫1各表中將數據進行聚合處理返回給調用接口。
返回數據
由於執行了兩次兩個羣組的新增數據,總共40條數據
分庫分表驗證成功!
問題記錄
springboot2.1.5默認使用mysql-connECTOR-java8.x的版本,這個版本的驅動使用shardingsphere時會有bug,推薦使用mysql-connector-java5.1.47版本