公司原先使用的數據庫中間件是mycat，但是mycat由於很早就開始不維護了，所以我們再使用過程中出現了很多問題都無法的到解決，如果各位同學有興趣可以再壓測時觀察下mycat的堆棧活動，ygc是非常頻繁的，並且很不穩定，當時考慮切換到Sharding-Proxy上面去，同樣是數據庫中間件這種方案我們內部認爲是改動最小的。
可是事實證明我們錯了，Sharding-Proxy由於也是剛開源不久的東西，其實也存在不少BUG，雖然社區活躍但是官方文檔實在太不全了，導致我們遇到每一個問題都需要花很大的精力去排查、定位、解決，所以我們最終還是放棄了Sharding-Proxy，準備使用Shardingjdbc，但是作爲一款客戶端直連的框架性能一定是最好的，只是業務可感知的改動量肯定也是最明顯的，例如是分片規則的定製化、多數據源切換（並非多分片的數據源，可能包含不需要shardingjdbc管理的數據源）、mycat原先的全局表、複雜子查詢的支持等等...
在這樣一個前提下，我們決定深度定製下shardingjdbc，讓業務無感知的直接從mycat切換到shardingjdbc上面
啓動初始化流程如下，由於YML配置無法滿足我們的需求，我們採用了Java代碼注入的方式：
之後源碼貼出來大家會發現我們apollo的配置也是非常簡潔，完全看不出和shardingjdbc有任何關係，這麼做也是爲了做到業務最小感知，當時也做了backup的方案，按項目一個個切換shardingjdbc，如果出問題那麼久通過apollo一個最簡單的開關完成回滾，整個項目加載流程如下：
下面貼出來整個的核心代碼吧，全部源碼實在是有點多無法一一列舉了，先看下我們封裝後的Apollo數據源配置吧：
#數據源，可通配，如果不存在10號分庫啓動時會自動過濾，所以前期配置可以放大一點，避免每次加一個分庫都需要去修改依次apollo
jdbc:mysql://xxxx/db_wh_shard_${01-10}
#backup 方案，是否開啓接入sharding
com.xxx.sharding.jdbc.datasource.isopen = false
#是否過濾小於當前日期的日誌表不加入shardingjbdc管理，節省內存，優化啓動時間
com.xxx.sharding.jdbc.table.log = true
初始化啓動核心類：
/**
* <p>Title:主數據源交給spring管理</p>
* <p>Description:infoSource</p>
*
* @return javax.sql.DataSource
* @throws
* @author QIQI
* @params [shardingDruidProperties]
* @date 2020/04/20 16:02
*/
@Bean(name = "dataSourceForSpring")
@Conditional(value = DataSardingConditional.class)
public LinkedHashMap<String, DataSource> dataSourceForSpring() {
shardingJDBCConfigEngine.getShardingJdbcDatasourceMap();
LinkedHashMap<String, DataSource> linkedHashMap = new LinkedHashMap<>();
linkedHashMap.putAll( shardingJDBCConfigEngine.SHARDING_JDBC_DATASOURCE_MAP() );
linkedHashMap.entrySet().removeIf( entry -> entry.getKey().contains( ShardingConstant.SHARD_SOURCE ) );
try {
//添加全局表數據源，兼容原先@DataSource 註解框架
linkedHashMap.forEach( (key, val) -> DynamicDataSourceContextHolder.dataSourceIds.add( key ) );
return linkedHashMap;
} catch (Exception e) {
throw new WmsShardingException( new ExceptionMessageImpl( ErrorMessage.DATASOURCE_ERROR.getCode() ) );
}
}
@Bean("dynamicDataSource")
@Conditional(value = DataSardingConditional.class)
public DynamicRoutingDataSource shardingDataSource(@Qualifier("dataSourceForSpring") LinkedHashMap<String, DataSource> dataSourceForSpring) throws SQLException {
DynamicRoutingDataSource dataSource = new DynamicRoutingDataSource();
DataSourceModel dataSourceModel = new DataSourceModel( shardingJDBCConfigEngine.SHARDING_JDBC_DATASOURCE_MAP(), false );
DataSourceChain.initHandler().handleWork( dataSourceModel );
databaseTableChain.initHandler().getShardingRuleConfiguration( WmsShardSourceEngine.getShardingDataSource() );
ShardingRuleConfiguration ruleConfiguration = DatabaseTableChain.shardingRuleConfiguration;
DatabaseTableChain.shardingRuleConfiguration = null; //釋放全局變量內存
//shardingDataSource create success
DataSource shardingDataSource = ShardingDataSourceFactory.createDataSource( WmsShardSourceEngine.getShardingDataSource(), ruleConfiguration, new Properties() );
Map<Object, Object> targetDataSources = new HashMap<>();
targetDataSources.put( ShardingConstant.SHARD_SOURCE, shardingDataSource );
targetDataSources.putAll( dataSourceForSpring );
targetDataSources.put( ShardingConstant.DEFAULT_SOURCE, targetDataSources.get( ShardingConstant.SHARD_SOURCE ) );
WmsShardSourceEngine.getShardingDataSource().putAll( dataSourceForSpring );
dataSource.setDataSourceMap( targetDataSources );
log.info( ShardingConstant.getWmsAppName() + "Shardingjdbc init And DataSourceMap init is Success " );
return dataSource;
}
@Bean
@ConfigurationProperties(prefix = "mybatis")
@Conditional(value = DataSardingConditional.class)
public MybatisSqlSessionFactoryBean sqlSessionFactory(@Qualifier("dynamicDataSource") DynamicRoutingDataSource dynamicRoutingDataSource) throws Exception {
MybatisSqlSessionFactoryBean bean = new MybatisSqlSessionFactoryBean();
bean.setDataSource( dynamicRoutingDataSource );
bean.setMapperLocations( new PathMatchingResourcePatternResolver().getResources( mybatisPlusMapper ) );
GlobalConfig globalConfig = new GlobalConfig();
globalConfig.setSqlInjector( new WmsSqlInjector() );
MybatisConfiguration mybatisConfiguration = new MybatisConfiguration();
mybatisConfiguration.setCacheEnabled( cache );
mybatisConfiguration.setLocalCacheScope( localCacheScope );
bean.setConfiguration( mybatisConfiguration );
bean.setGlobalConfig( globalConfig );
bean.setPlugins( new PaginationInterceptor(),new OptimisticLockerInterceptor() );
return bean;
}
@Bean(name = "sqlSessionTemplate")
@Conditional(value = DataSardingConditional.class)
@Primary
public SqlSessionTemplate sqlSessionTemplate(@Qualifier("sqlSessionFactory") SqlSessionFactory sqlSessionFactory) {
return new SqlSessionTemplate( sqlSessionFactory );
}
@Bean
@Conditional(value = DataSardingConditional.class)
public DataSourceTransactionManager transactitonManager(@Qualifier("dynamicDataSource") DynamicRoutingDataSource dynamicRoutingDataSource) {
return new DataSourceTransactionManager( dynamicRoutingDataSource );
}
Shardingjdbc 常用類說明
| 期望實現 | 類 | 代碼摘要 |
| 如何添加分片配置信息 | StandardShardingStrategyConfiguration |
代碼摘要V1.1 |
| 單分片查詢下，如何支持複雜SQL，包含子查詢（其實shardingjdbc原生支持的，但是默認是按照多分片解析的，所以你會發現你複雜的子查詢會報錯） |
|
代碼摘要V1.2 |
| shardingjdbc中怎麼設置廣播表 | setBroadcastTables |
代碼摘要V1.3 |
代碼摘要V1.1：
/**
* <p>Title:簡化單庫下面所有表應用的分片規則配置</p>
* <p>Description:目前單庫所有表規則都是一致的，就不需要循環所有配置了</p>
*
* @return org.apache.shardingsphere.api.config.sharding.ShardingRuleConfiguration
* @throws
* @author QIQI
* @params [map]
* @date 2020/04/14 14:15
*/
public void getShardingRuleConfiguration(final Map<String, DataSource> map) throws SQLException {
List<TableRuleConfiguration> configurationList = new ArrayList<>();
StandardShardingStrategyConfiguration standardShardingStrategyConfiguration =
new StandardShardingStrategyConfiguration( ShardingConstant.OUID, new WmsShardPreciseShardingAlgorithm() );
//表結構信息,優化啓動時間，只取出第一個符合條件的數據源即可
Optional<Map.Entry<String, DataSource>> sourceEntry = map.entrySet().stream().
filter( entry -> entry.getKey().contains( ShardingConstant.SHARD_SOURCE ) ).findFirst();
final List<TableRouteModel> tableShardRouteModelList = new ArrayList<>();
sourceEntry.ifPresent( entry -> {
DruidDataSource druidDataSource = (DruidDataSource) sourceEntry.get().getValue();
try {
tableShardRouteModelList.addAll( tableRouteService.getShardTablesList( druidDataSource,
WmsShardCustomizationEngine.getNotInSql( shardingjdbcGlobalProperties.getGlobal() ) ) );
WmsShardCustomizationEngine.logTableReduce( tableShardRouteModelList );
//執行分片表邏輯添加
tableShardRouteModelList.forEach( v -> {
TableRuleConfiguration tableRuleConfiguration = new TableRuleConfiguration( v.getTableName() );
tableRuleConfiguration.setDatabaseShardingStrategyConfig( standardShardingStrategyConfiguration );
configurationList.add( tableRuleConfiguration );
} );
DatabaseTableChain.shardingRuleConfiguration.setTableRuleConfigs( configurationList );
log.info( ShardingConstant.getWmsAppName() + "ShardDatabaseTablesHandler ShardDatabaseTablesHandler init is success" );
} catch (SQLException e) {
log.warn( "ShardDatabaseTablesHandler getShardingRuleConfiguration is error", e );
}
} );
next( map );
}
代碼摘要V1.2：
public static List<String> bindWmsShardingTable(List<String> logicTableNameList){
List<String> tableRules = ImmutableList.of(
String.join( ",",logicTableNameList )
);
return tableRules;
}
//單片分組綁定
DatabaseTableChain.shardingRuleConfiguration.setBindingTableGroups
( WmsShardCustomizationEngine.bindWmsShardingTable( logicTables ) );
代碼摘要V1.3：
DatabaseTableChain.shardingRuleConfiguration.setBroadcastTables
( WmsShardCustomizationEngine.broadcastTables( shardingjdbcGlobalProperties.getGlobal() ) );