redis的兩種集羣模式

一、哨兵模式和集羣模式 從節點 的區別

• 哨兵模式中的從節點默認是可讀的。
• 集羣模式中的從節點默認是不可讀的，只是主節點的熱備。
  如果想要從集羣模式中的從節點讀取數據，需要客戶端在建立連接之後先發送一個readonly命令。

二、lettuce對集羣模式的處理

1. 建立連接後，先查詢集羣信息，緩存哈希分槽，以後如果沒有拓撲刷新就直接使用緩存的哈希分槽定位機器，不會每次隨便連一個，等着服務器重定向；

io/lettuce/core/cluster/models/partitions/Partitions.java

 	/**
     * Update the partition cache. Updates are necessary after the partition details have changed.
     */
    public void updateCache() {

        synchronized (partitions) {

            if (partitions.isEmpty()) {
                this.slotCache = EMPTY;
                this.nodeReadView = Collections.emptyList();
                return;
            }

            RedisClusterNode[] slotCache = new RedisClusterNode[SlotHash.SLOT_COUNT];
            List<RedisClusterNode> readView = new ArrayList<>(partitions.size());

            for (RedisClusterNode partition : partitions) {

                readView.add(partition);
                partition.forEachSlot(i -> slotCache[i] = partition);
            }

            this.slotCache = slotCache;
            this.nodeReadView = Collections.unmodifiableCollection(readView);
        }
    }

2. 如果連接的是從節點，會先發送一個readonly命令，然後這個連接以後就可以從從節點讀取數據了；

io/lettuce/core/cluster/PooledClusterConnectionProvider.java

		@Override
        public ConnectionFuture<StatefulRedisConnection<K, V>> apply(ConnectionKey key) {

            if (key.nodeId != null && getPartitions().getPartitionByNodeId(key.nodeId) == null) {
                clusterEventListener.onUnknownNode();
                throw connectionAttemptRejected("node id " + key.nodeId);
            }

            if (key.host != null && partitions.getPartition(key.host, key.port) == null) {
                clusterEventListener.onUnknownNode();
                if (validateClusterNodeMembership()) {
                    throw connectionAttemptRejected(key.host + ":" + key.port);
                }
            }

            ConnectionFuture<StatefulRedisConnection<K, V>> connection = delegate.apply(key);

            LettuceAssert.notNull(connection, "Connection is null. Check ConnectionKey because host and nodeId are null.");

            if (key.intent == Intent.READ) {

                connection = connection.thenCompose(c -> {
					// 發送readonly命令
                    RedisFuture<String> stringRedisFuture = c.async().readOnly();
                    return stringRedisFuture.thenApply(s -> c).whenCompleteAsync((s, throwable) -> {
                        if (throwable != null) {
                            c.close();
                        }
                    });
                });
            }

            connection = connection.thenApply(c -> {
                synchronized (stateLock) {
                    c.setAutoFlushCommands(autoFlushCommands);
                }
                return c;
            });

            return connection;
        }
    }

3. 執行向redis發送命令之前，先用哈希分槽判斷應該發往那些節點，這時最少能選出一主一從2個可用節點，再判斷命令類型，如果是寫命令，那就只能連接主節點，如果是讀命令，則根據select返回的可用節點進行選擇，如果設置了順序敏感就選第一個，如果沒有設置就隨機選一個。

io/lettuce/core/cluster/PooledClusterConnectionProvider.java

	/**
	 *	根據命令類型選擇讀鏈接或寫鏈接
	 */
	@Override
    public CompletableFuture<StatefulRedisConnection<K, V>> getConnectionAsync(Intent intent, int slot) {

        if (debugEnabled) {
            logger.debug("getConnection(" + intent + ", " + slot + ")");
        }

        if (intent == Intent.READ && readFrom != null && readFrom != ReadFrom.MASTER) {
            return getReadConnection(slot);
        }

        return getWriteConnection(slot).toCompletableFuture();
    }

	/**
	 *	寫鏈接只返回master節點
	 */
    private CompletableFuture<StatefulRedisConnection<K, V>> getWriteConnection(int slot) {

        CompletableFuture<StatefulRedisConnection<K, V>> writer;// avoid races when reconfiguring partitions.
        synchronized (stateLock) {
            writer = writers[slot];
        }

        if (writer == null) {
            RedisClusterNode partition = partitions.getPartitionBySlot(slot);
            if (partition == null) {
                clusterEventListener.onUncoveredSlot(slot);
                return Futures.failed(new PartitionSelectorException("Cannot determine a partition for slot " + slot + ".",
                        partitions.clone()));
            }

            // Use always host and port for slot-oriented operations. We don't want to get reconnected on a different
            // host because the nodeId can be handled by a different host.
            RedisURI uri = partition.getUri();
            ConnectionKey key = new ConnectionKey(Intent.WRITE, uri.getHost(), uri.getPort());

            ConnectionFuture<StatefulRedisConnection<K, V>> future = getConnectionAsync(key);

            return future.thenApply(connection -> {

                synchronized (stateLock) {
                    if (writers[slot] == null) {
                        writers[slot] = CompletableFuture.completedFuture(connection);
                    }
                }

                return connection;
            }).toCompletableFuture();
        }

        return writer;
    }

	/**
	 *	讀鏈接從select返回的可用節點（包括master節點和slave節點）進行選擇，如果isOrderSensitive就返回第一個，否則隨機返回一個
	 */
    private CompletableFuture<StatefulRedisConnection<K, V>> getReadConnection(int slot) {

        CompletableFuture<StatefulRedisConnection<K, V>> readerCandidates[];// avoid races when reconfiguring partitions.

        boolean cached = true;

        synchronized (stateLock) {
            readerCandidates = readers[slot];
        }

        if (readerCandidates == null) {

            RedisClusterNode master = partitions.getPartitionBySlot(slot);
            if (master == null) {
                clusterEventListener.onUncoveredSlot(slot);
                return Futures.failed(new PartitionSelectorException(String.format(
                        "Cannot determine a partition to read for slot %d.", slot), partitions.clone()));
            }

            List<RedisNodeDescription> candidates = getReadCandidates(master);
            List<RedisNodeDescription> selection = readFrom.select(new ReadFrom.Nodes() {
                @Override
                public List<RedisNodeDescription> getNodes() {
                    return candidates;
                }

                @Override
                public Iterator<RedisNodeDescription> iterator() {
                    return candidates.iterator();
                }
            });

            if (selection.isEmpty()) {
                clusterEventListener.onUncoveredSlot(slot);
                return Futures.failed(new PartitionSelectorException(String.format(
                        "Cannot determine a partition to read for slot %d with setting %s.", slot, readFrom), partitions
                        .clone()));
            }

            readerCandidates = getReadFromConnections(selection);
            cached = false;
        }

        CompletableFuture<StatefulRedisConnection<K, V>> selectedReaderCandidates[] = readerCandidates;

        if (cached) {

            return CompletableFuture.allOf(readerCandidates).thenCompose(
                    v -> {

                        boolean orderSensitive = isOrderSensitive(selectedReaderCandidates);

                        if (!orderSensitive) {

                            CompletableFuture<StatefulRedisConnection<K, V>> candidate = findRandomActiveConnection(
                                    selectedReaderCandidates, Function.identity());

                            if (candidate != null) {
                                return candidate;
                            }
                        }

                        for (CompletableFuture<StatefulRedisConnection<K, V>> candidate : selectedReaderCandidates) {

                            if (candidate.join().isOpen()) {
                                return candidate;
                            }
                        }

                        return selectedReaderCandidates[0];
                    });
        }

        CompletableFuture<StatefulRedisConnection<K, V>[]> filteredReaderCandidates = new CompletableFuture<>();

        CompletableFuture.allOf(readerCandidates).thenApply(v -> selectedReaderCandidates)
                .whenComplete((candidates, throwable) -> {

                    if (throwable == null) {
                        filteredReaderCandidates.complete(getConnections(candidates));
                        return;
                    }

                    StatefulRedisConnection<K, V>[] connections = getConnections(selectedReaderCandidates);

                    if (connections.length == 0) {
                        filteredReaderCandidates.completeExceptionally(throwable);
                        return;
                    }

                    filteredReaderCandidates.complete(connections);
                });

        return filteredReaderCandidates
                .thenApply(statefulRedisConnections -> {

                    boolean orderSensitive = isOrderSensitive(statefulRedisConnections);

                    CompletableFuture<StatefulRedisConnection<K, V>> toCache[] = new CompletableFuture[statefulRedisConnections.length];

                    for (int i = 0; i < toCache.length; i++) {
                        toCache[i] = CompletableFuture.completedFuture(statefulRedisConnections[i]);
                    }
                    synchronized (stateLock) {
                        readers[slot] = toCache;
                    }

                    if (!orderSensitive) {

                        StatefulRedisConnection<K, V> candidate = findRandomActiveConnection(selectedReaderCandidates,
                                CompletableFuture::join);

                        if (candidate != null) {
                            return candidate;
                        }
                    }

                    for (StatefulRedisConnection<K, V> candidate : statefulRedisConnections) {
                        if (candidate.isOpen()) {
                            return candidate;
                        }
                    }

                    return statefulRedisConnections[0];
                });
    }

    private boolean isOrderSensitive(Object[] connections) {
        return OrderingReadFromAccessor.isOrderSensitive(readFrom) || connections.length == 1;
    }