sharding-jdbc之路由引擎

根據解析上下文匹配數據庫和表的分片策略，並生成路由路徑。對於攜帶分片鍵的SQL，根據分片鍵的不同可以劃分爲單片路由(分片鍵的操作符是等號)、多片路由(分片鍵的操作符是IN)和範圍路由(分片鍵的操作符是BETWEEN)。不攜帶分片鍵的SQL則採用廣播路由。

分片策略通常可以採用由數據庫內置或由用戶方配置。數據庫內置的方案較爲簡單，內置的分片策略大致可分爲尾數取模、哈希、範圍、標籤、時間等。由用戶方配置的分片策略則更加靈活，可以根據使用方需求定製複合分片策略。如果配合數據自動遷移來使用，可以做到無需用戶關注分片策略，自動由數據庫中間層分片和平衡數據即可，進而做到使分佈式數據庫具有的彈性伸縮的能力。在ShardingSphere的線路規劃中，彈性伸縮將於4.x開啓。

路由引擎的整體結構劃分如下圖

分片路由

分片路由又可分爲直接路由、標準路由和笛卡爾積路由這3種類型。

直接路由

滿足直接路由的條件相對苛刻，它需要通過Hint（使用HintAPI直接指定路由至庫表）方式分片，並且是只分庫不分表的前提下，則可以避免SQL解析和之後的結果歸併。因此它的兼容性最好，可以執行包括子查詢、自定義函數等複雜情況的任意SQL。直接路由還可以用於分片鍵不在SQL中的場景。例如，設置用於數據庫分片的鍵爲3：

hintManager.setDatabaseShardingValue(3);

假如路由算法爲value % 2，當一個邏輯庫t_order對應2個真實庫t_order_0和t_order_1時，路由後SQL將在t_order_1上執行。下方是使用API的代碼樣例：

String sql = "SELECT * FROM t_order";
try (
        HintManager hintManager = HintManager.getInstance();
        Connection conn = dataSource.getConnection();
        PreparedStatement pstmt = conn.prepareStatement(sql)) {
    hintManager.setDatabaseShardingValue(3);
    try (ResultSet rs = pstmt.executeQuery()) {
        while (rs.next()) {
            //...
        }
    }
}

直接路由源碼：

@RequiredArgsConstructor
public final class DatabaseHintRoutingEngine implements RoutingEngine {
    //所有分庫的數據庫名稱集合，由於直接路由只支持分庫，不支持分表，不需要維護分表的集合
    private final Collection<String> dataSourceNames;
    //直接路由使用Hint路由策略進行分庫
    private final HintShardingStrategy databaseShardingStrategy;
    
    @Override
    public RoutingResult route() {
    	 //獲取數據庫分片值
        Optional<ShardingValue> shardingValue = HintManagerHolder.getDatabaseShardingValue(HintManagerHolder.DB_TABLE_NAME);
        // 校驗是否存在值
        Preconditions.checkState(shardingValue.isPresent());
        Collection<String> routingDataSources;
        //得到分片結果，Hint分片策略
        routingDataSources = databaseShardingStrategy.doSharding(dataSourceNames, Collections.singletonList(shardingValue.get()));
        Preconditions.checkState(!routingDataSources.isEmpty(), "no database route info");
        RoutingResult result = new RoutingResult();
        //遍歷分片結果，填充到路由結果中
        for (String each : routingDataSources) {
        //路由表單元集合，主要存儲數據庫名和路由表（包含邏輯表和實際表名稱）返回這個結果便於在下一步的改寫中，指定數據庫和實際表的sql
            result.getTableUnits().getTableUnits().add(new TableUnit(each));
        }
        return result;
    }
}

標準路由

標準路由是ShardingSphere最爲推薦使用的分片方式，它的適用範圍是不包含關聯查詢或僅包含綁定表之間關聯查詢的SQL。當分片運算符是等於號時，路由結果將落入單庫（表），當分片運算符是BETWEEN或IN時，則路由結果不一定落入唯一的庫（表），因此一條邏輯SQL最終可能被拆分爲多條用於執行的真實SQL。舉例說明，如果按照order_id的奇數和偶數進行數據分片，一個單表查詢的SQL如下：

SELECT * FROM t_order WHERE order_id IN (1, 2);

那麼路由的結果應爲：

SELECT * FROM t_order_0 WHERE order_id IN (1, 2);
SELECT * FROM t_order_1 WHERE order_id IN (1, 2);

綁定表的關聯查詢與單表查詢複雜度和性能相當。舉例說明，如果一個包含綁定表的關聯查詢的SQL如下：

SELECT * FROM t_order o JOIN t_order_item i ON o.order_id=i.order_id  WHERE order_id IN (1, 2);

那麼路由的結果應爲：

SELECT * FROM t_order_0 o JOIN t_order_item_0 i ON o.order_id=i.order_id  WHERE order_id IN (1, 2);
SELECT * FROM t_order_1 o JOIN t_order_item_1 i ON o.order_id=i.order_id  WHERE order_id IN (1, 2);

標準路由源碼：

@RequiredArgsConstructor
public final class StandardRoutingEngine implements RoutingEngine {
    //分片策略接口，具體實現有外部指定
    private final ShardingRule shardingRule;
    //邏輯表
    private final String logicTableName;
    //分片條件
    private final ShardingConditions shardingConditions;
   
    @Override
    public RoutingResult route() {
        return generateRoutingResult(getDataNodes(shardingRule.getTableRuleByLogicTableName(logicTableName)));
    }
     /**
     * 生成路由結果
     * DataNode包含如下三個屬性
     * private static final String DELIMITER = ".";
     *
     * private final String dataSourceName;
     *
     * private final String tableName;
     */
    private RoutingResult generateRoutingResult(final Collection<DataNode> routedDataNodes) {
        RoutingResult result = new RoutingResult();
        //與直接路由中相同，最終都是組裝路由表單元集合
        for (DataNode each : routedDataNodes) {
        	//創建路由表單元實例，並指定數據庫名
            TableUnit tableUnit = new TableUnit(each.getDataSourceName());
            //賦值路由表 邏輯表和實際表
            tableUnit.getRoutingTables().add(new RoutingTable(logicTableName, each.getTableName()));
            result.getTableUnits().getTableUnits().add(tableUnit);
        }
        return result;
    }
    //獲取DataNode節點集合，參數爲表規則配置
    private Collection<DataNode> getDataNodes(final TableRule tableRule) {
    	//判斷是否是指定表的規則
        if (isRoutingByHint(tableRule)) {
            return routeByHint(tableRule);
        }
        //是否根據分片條件路由
        if (isRoutingByShardingConditions(tableRule)) {
            return routeByShardingConditions(tableRule);
        }
        //沒有路由條件，則使用混合方式
        return routeByMixedConditions(tableRule);
    }
    
    private boolean isRoutingByHint(final TableRule tableRule) {
    	//數據庫分片策略是Hint分片策略 且表分片策略也是Hint分片策略
        return shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy && shardingRule.getTableShardingStrategy(tableRule) instanceof HintShardingStrategy;
    }
    //Hint路由方式
    private Collection<DataNode> routeByHint(final TableRule tableRule) {
        return route(tableRule, getDatabaseShardingValuesFromHint(), getTableShardingValuesFromHint());
    }
    
    private boolean isRoutingByShardingConditions(final TableRule tableRule) {
     	//數據庫和表的分片策略都是不是Hint分片方式
        return !(shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy || shardingRule.getTableShardingStrategy(tableRule) instanceof HintShardingStrategy);
    }
     //分片條件路由方式
    private Collection<DataNode> routeByShardingConditions(final TableRule tableRule) {
        return shardingConditions.getShardingConditions().isEmpty() ? route(tableRule, Collections.<ShardingValue>emptyList(), Collections.<ShardingValue>emptyList())
                : routeByShardingConditionsWithCondition(tableRule);
    }
    //路由條件不爲空，根據表分片規則進行路由
    private Collection<DataNode> routeByShardingConditionsWithCondition(final TableRule tableRule) {
        Collection<DataNode> result = new LinkedList<>();
        for (ShardingCondition each : shardingConditions.getShardingConditions()) {
            Collection<DataNode> dataNodes = route(tableRule, getShardingValuesFromShardingConditions(shardingRule.getDatabaseShardingStrategy(tableRule).getShardingColumns(), each),
                    getShardingValuesFromShardingConditions(shardingRule.getTableShardingStrategy(tableRule).getShardingColumns(), each));
            reviseShardingConditions(each, dataNodes);
            result.addAll(dataNodes);
        }
        return result;
    }
    
    private Collection<DataNode> routeByMixedConditions(final TableRule tableRule) {
        return shardingConditions.getShardingConditions().isEmpty() ? routeByMixedConditionsWithHint(tableRule) : routeByMixedConditionsWithCondition(tableRule);
    }
    
    private Collection<DataNode> routeByMixedConditionsWithCondition(final TableRule tableRule) {
        Collection<DataNode> result = new LinkedList<>();
        for (ShardingCondition each : shardingConditions.getShardingConditions()) {
            Collection<DataNode> dataNodes = route(tableRule, getDatabaseShardingValues(tableRule, each), getTableShardingValues(tableRule, each));
            reviseShardingConditions(each, dataNodes);
            result.addAll(dataNodes);
        }
        return result;
    }
    
    private Collection<DataNode> routeByMixedConditionsWithHint(final TableRule tableRule) {
        if (shardingRule.getDatabaseShardingStrategy(tableRule) instanceof HintShardingStrategy) {
            return route(tableRule, getDatabaseShardingValuesFromHint(), Collections.<ShardingValue>emptyList());
        }
        return route(tableRule, Collections.<ShardingValue>emptyList(), getTableShardingValuesFromHint());
    }
    //獲取數據庫分片值
    private List<ShardingValue> getDatabaseShardingValues(final TableRule tableRule, final ShardingCondition shardingCondition) {
        ShardingStrategy dataBaseShardingStrategy = shardingRule.getDatabaseShardingStrategy(tableRule);
        return isGettingShardingValuesFromHint(dataBaseShardingStrategy)
                ? getDatabaseShardingValuesFromHint() : getShardingValuesFromShardingConditions(dataBaseShardingStrategy.getShardingColumns(), shardingCondition);
    }
    
    private List<ShardingValue> getTableShardingValues(final TableRule tableRule, final ShardingCondition shardingCondition) {
        ShardingStrategy tableShardingStrategy = shardingRule.getTableShardingStrategy(tableRule);
        return isGettingShardingValuesFromHint(tableShardingStrategy)
                ? getTableShardingValuesFromHint() : getShardingValuesFromShardingConditions(tableShardingStrategy.getShardingColumns(), shardingCondition);
    }
    
    private boolean isGettingShardingValuesFromHint(final ShardingStrategy shardingStrategy) {
        return shardingStrategy instanceof HintShardingStrategy;
    }
    
    private List<ShardingValue> getDatabaseShardingValuesFromHint() {
        Optional<ShardingValue> shardingValueOptional = HintManagerHolder.getDatabaseShardingValue(logicTableName);
        return shardingValueOptional.isPresent() ? Collections.singletonList(shardingValueOptional.get()) : Collections.<ShardingValue>emptyList();
    }
    
    private List<ShardingValue> getTableShardingValuesFromHint() {
        Optional<ShardingValue> shardingValueOptional = HintManagerHolder.getTableShardingValue(logicTableName);
        return shardingValueOptional.isPresent() ? Collections.singletonList(shardingValueOptional.get()) : Collections.<ShardingValue>emptyList();
    }
    
    private List<ShardingValue> getShardingValuesFromShardingConditions(final Collection<String> shardingColumns, final ShardingCondition shardingCondition) {
        List<ShardingValue> result = new ArrayList<>(shardingColumns.size());
        for (ShardingValue each : shardingCondition.getShardingValues()) {
            Optional<BindingTableRule> bindingTableRule = shardingRule.findBindingTableRule(logicTableName);
            if ((logicTableName.equals(each.getLogicTableName()) || bindingTableRule.isPresent() && bindingTableRule.get().hasLogicTable(logicTableName)) 
                    && shardingColumns.contains(each.getColumnName())) {
                result.add(each);
            }
        }
        return result;
    }
    
    private Collection<DataNode> route(final TableRule tableRule, final List<ShardingValue> databaseShardingValues, final List<ShardingValue> tableShardingValues) {
        Collection<String> routedDataSources = routeDataSources(tableRule, databaseShardingValues);
        Collection<DataNode> result = new LinkedList<>();
        for (String each : routedDataSources) {
            result.addAll(routeTables(tableRule, each, tableShardingValues));
        }
        return result;
    }
    
    private Collection<String> routeDataSources(final TableRule tableRule, final List<ShardingValue> databaseShardingValues) {
        Collection<String> availableTargetDatabases = tableRule.getActualDatasourceNames();
        if (databaseShardingValues.isEmpty()) {
            return availableTargetDatabases;
        }
        Collection<String> result = new LinkedHashSet<>(shardingRule.getDatabaseShardingStrategy(tableRule).doSharding(availableTargetDatabases, databaseShardingValues));
        Preconditions.checkState(!result.isEmpty(), "no database route info");
        return result;
    }
    
    private Collection<DataNode> routeTables(final TableRule tableRule, final String routedDataSource, final List<ShardingValue> tableShardingValues) {
        Collection<String> availableTargetTables = tableRule.getActualTableNames(routedDataSource);
        Collection<String> routedTables = new LinkedHashSet<>(tableShardingValues.isEmpty() ? availableTargetTables
                : shardingRule.getTableShardingStrategy(tableRule).doSharding(availableTargetTables, tableShardingValues));
        Preconditions.checkState(!routedTables.isEmpty(), "no table route info");
        Collection<DataNode> result = new LinkedList<>();
        for (String each : routedTables) {
            result.add(new DataNode(routedDataSource, each));
        }
        return result;
    }
    
    private void reviseShardingConditions(final ShardingCondition each, final Collection<DataNode> dataNodes) {
        if (each instanceof InsertShardingCondition) {
            ((InsertShardingCondition) each).getDataNodes().addAll(dataNodes);
        }
    }
}

笛卡爾路由

笛卡爾路由是最複雜的情況，它無法根據綁定表的關係定位分片規則，因此非綁定表之間的關聯查詢需要拆解爲笛卡爾積組合執行。如果上個示例中的SQL並未配置綁定表關係，那麼路由的結果應爲：

SELECT * FROM t_order_0 o JOIN t_order_item_0 i ON o.order_id=i.order_id  WHERE order_id IN (1, 2);
SELECT * FROM t_order_0 o JOIN t_order_item_1 i ON o.order_id=i.order_id  WHERE order_id IN (1, 2);
SELECT * FROM t_order_1 o JOIN t_order_item_0 i ON o.order_id=i.order_id  WHERE order_id IN (1, 2);
SELECT * FROM t_order_1 o JOIN t_order_item_1 i ON o.order_id=i.order_id  WHERE order_id IN (1, 2);

笛卡爾路由使用頻率很低，就不做具體分析了

廣播路由

對於不攜帶分片鍵的SQL，則採取廣播路由的方式。根據SQL類型又可以劃分爲全庫表路由、全庫路由、全實例路由、單播路由和阻斷路由這5種類型。

全庫表路由

全庫表路由用於處理對數據庫中與其邏輯表相關的所有真實表的操作，主要包括不帶分片鍵的DQL和DML，以及DDL等。例如：

SELECT * FROM t_order WHERE good_prority IN (1, 10);

則會遍歷所有數據庫中的所有表，逐一匹配邏輯表和真實表名，能夠匹配得上則執行。路由後成爲

SELECT * FROM t_order_0 WHERE good_prority IN (1, 10);
SELECT * FROM t_order_1 WHERE good_prority IN (1, 10);
SELECT * FROM t_order_2 WHERE good_prority IN (1, 10);
SELECT * FROM t_order_3 WHERE good_prority IN (1, 10);

全庫表路由源碼：

@RequiredArgsConstructor
public final class TableBroadcastRoutingEngine implements RoutingEngine {
    
    private final ShardingRule shardingRule;
    //sql語句
    private final SQLStatement sqlStatement;
    
    @Override
    public RoutingResult route() {
        RoutingResult result = new RoutingResult();
        for (String each : getLogicTableNames()) {
            result.getTableUnits().getTableUnits().addAll(getAllTableUnits(each));
        }
        return result;
    }
    //湖區哦v所有邏輯表
    private Collection<String> getLogicTableNames() {
        if (isOperateIndexWithoutTable()) {
        	//通過分片策略獲取邏輯表
            return Collections.singletonList(shardingRule.getLogicTableName(getIndexToken().getIndexName()));
        }
        return sqlStatement.getTables().getTableNames();
    }
    
    private boolean isOperateIndexWithoutTable() {
        return sqlStatement instanceof DDLStatement && sqlStatement.getTables().isEmpty();
    }
    //獲取Token的索引
    private IndexToken getIndexToken() {
        List<SQLToken> sqlTokens = sqlStatement.getSQLTokens();
        Preconditions.checkState(1 == sqlTokens.size());
        return (IndexToken) sqlTokens.get(0);
    }
    //根據邏輯表獲取實際表，並封裝爲分片表單元實例
    private Collection<TableUnit> getAllTableUnits(final String logicTableName) {
        Collection<TableUnit> result = new LinkedList<>();
        TableRule tableRule = shardingRule.getTableRuleByLogicTableName(logicTableName);
        for (DataNode each : tableRule.getActualDataNodes()) {
            TableUnit tableUnit = new TableUnit(each.getDataSourceName());
            tableUnit.getRoutingTables().add(new RoutingTable(logicTableName, each.getTableName()));
            result.add(tableUnit);
        }
        return result;
    }
}

全庫路由

全庫路由用於處理對數據庫的操作，包括用於庫設置的SET類型的數據庫管理命令，以及TCL這樣的事務控制語句。在這種情況下，會根據邏輯庫的名字遍歷所有符合名字匹配的真實庫，並在真實庫中執行該命令，例如：

SET autocommit=0;

在t_order中執行，t_order有2個真實庫。則實際會在t_order_0和t_order_1上都執行這個命令。

@RequiredArgsConstructor
public final class DatabaseBroadcastRoutingEngine implements RoutingEngine {
    
    private final ShardingRule shardingRule;
    
    @Override
    public RoutingResult route() {
        RoutingResult result = new RoutingResult();
        for (String each : shardingRule.getShardingDataSourceNames().getDataSourceNames()) {
            result.getTableUnits().getTableUnits().add(new TableUnit(each));
        }
        return result;
    }
}

全實例路由

全實例路由用於DCL操作，授權語句針對的是數據庫的實例。無論一個實例中包含多少個Schema，每個數據庫的實例只執行一次。例如：CREATE USER [email protected] identified BY '123';
這個命令將在所有的真實數據庫實例中執行，以確保customer用戶可以訪問每一個實例。

@RequiredArgsConstructor
public final class InstanceBroadcastRoutingEngine implements RoutingEngine {
    
    private final ShardingRule shardingRule;
    
    private final ShardingDataSourceMetaData shardingDataSourceMetaData;
    
    @Override
    public RoutingResult route() {
        RoutingResult result = new RoutingResult();
        for (String each : shardingRule.getShardingDataSourceNames().getDataSourceNames()) {
            if (shardingDataSourceMetaData.getAllInstanceDataSourceNames().contains(each)) {
                result.getTableUnits().getTableUnits().add(new TableUnit(each));
            }
        }
        return result;
    }
}

單播路由

單播路由用於獲取某一真實表信息的場景，它僅需要從任意庫中的任意真實表中獲取數據即可。例如：

DESCRIBE t_order;

t_order的兩個真實表t_order_0，t_order_1的描述結構相同，所以這個命令在任意真實表上選擇執行一次。

@RequiredArgsConstructor
public final class UnicastRoutingEngine implements RoutingEngine {
    
    private final ShardingRule shardingRule;
    
    private final Collection<String> logicTables;
    
    @Override
    public RoutingResult route() {
        RoutingResult result = new RoutingResult();
        if (shardingRule.isAllBroadcastTables(logicTables)) {
            List<RoutingTable> routingTables = new ArrayList<>(logicTables.size());
            for (String each : logicTables) {
                routingTables.add(new RoutingTable(each, each));
            }
            TableUnit tableUnit = new TableUnit(shardingRule.getShardingDataSourceNames().getDataSourceNames().iterator().next());
            tableUnit.getRoutingTables().addAll(routingTables);
            result.getTableUnits().getTableUnits().add(tableUnit);
        } else if (logicTables.isEmpty()) {
            result.getTableUnits().getTableUnits().add(new TableUnit(shardingRule.getShardingDataSourceNames().getDataSourceNames().iterator().next()));
        } else if (1 == logicTables.size()) {
            String logicTableName = logicTables.iterator().next();
            DataNode dataNode = shardingRule.findDataNode(logicTableName);
            TableUnit tableUnit = new TableUnit(dataNode.getDataSourceName());
            tableUnit.getRoutingTables().add(new RoutingTable(logicTableName, dataNode.getTableName()));
            result.getTableUnits().getTableUnits().add(tableUnit);
        } else {
            String dataSourceName = null;
            List<RoutingTable> routingTables = new ArrayList<>(logicTables.size());
            for (String each : logicTables) {
                DataNode dataNode = shardingRule.findDataNode(dataSourceName, each);
                routingTables.add(new RoutingTable(each, dataNode.getTableName()));
                if (null == dataSourceName) {
                    dataSourceName = dataNode.getDataSourceName();
                }
            }
            TableUnit tableUnit = new TableUnit(dataSourceName);
            tableUnit.getRoutingTables().addAll(routingTables);
            result.getTableUnits().getTableUnits().add(tableUnit);
        }
        return result;
    }
}

阻斷路由

阻斷路由用於屏蔽SQL對數據庫的操作，例如：

USE order_db;

這個命令不會在真實數據庫中執行，因爲ShardingSphere採用的是邏輯Schema的方式，無需將切換數據庫Schema的命令發送至數據庫中。

public final class IgnoreRoutingEngine implements RoutingEngine {
    
    @Override
    public RoutingResult route() {
        return new RoutingResult();
    }
}

sharding-jdbc之路由引擎

分片路由

直接路由

標準路由

笛卡爾路由

廣播路由

全庫表路由

全庫路由

全實例路由

單播路由

阻斷路由

如何使用 JS 判斷用戶是否處於活躍狀態

通過HPA+CronHPA組合應對業務複雜彈性伸縮場景

SpringBoot--時間校驗

觀察者模式在Spring中的應用

代理模式在Spring中的應用

Sharding-jdbc之分片

Mybatis映射集Mappers解析

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結