整體概述
Redis 一個服務器可以和多個客戶端建立網絡連接,每個客戶端都可以向服務器發送命令請求,服務器接收客戶端的命令,處理後將結果返回給客戶端。
Redis 的文件事件處理器使用 I/O 多路複用
,Redis 使用單線程單進程
處理命令請求,與多個客戶端進行網絡通信。
每個連接了 Redis 服務器的客戶端,服務器都建立了一個 redisClient
結構的客戶端狀態
,保存了客戶端當前的狀態信息,以及執行相關功能時用到的數據結構。
Redis 服務器狀態結構的 clients 屬性是一個鏈表,保存了所有與服務器連接的客戶端狀態。
struct redisServer {
……
// 保存了所有客戶端狀態的鏈表
list *clients;
……
};
客戶端屬性
先貼一下 client 完整的數據結構:
typedef struct client {
uint64_t id; /* Client incremental unique ID. */
connection *conn;
int resp; /* RESP protocol version. Can be 2 or 3. */
redisDb *db; /* Pointer to currently SELECTed DB. */
robj *name; /* As set by CLIENT SETNAME. */
sds querybuf; /* Buffer we use to accumulate client queries. */
size_t qb_pos; /* The position we have read in querybuf. */
sds pending_querybuf; /* If this client is flagged as master, this buffer
represents the yet not applied portion of the
replication stream that we are receiving from
the master. */
size_t querybuf_peak; /* Recent (100ms or more) peak of querybuf size. */
int argc; /* Num of arguments of current command. */
robj **argv; /* Arguments of current command. */
int original_argc; /* Num of arguments of original command if arguments were rewritten. */
robj **original_argv; /* Arguments of original command if arguments were rewritten. */
size_t argv_len_sum; /* Sum of lengths of objects in argv list. */
struct redisCommand *cmd, *lastcmd; /* Last command executed. */
user *user; /* User associated with this connection. If the
user is set to NULL the connection can do
anything (admin). */
int reqtype; /* Request protocol type: PROTO_REQ_* */
int multibulklen; /* Number of multi bulk arguments left to read. */
long bulklen; /* Length of bulk argument in multi bulk request. */
list *reply; /* List of reply objects to send to the client. */
unsigned long long reply_bytes; /* Tot bytes of objects in reply list. */
size_t sentlen; /* Amount of bytes already sent in the current
buffer or object being sent. */
time_t ctime; /* Client creation time. */
long duration; /* Current command duration. Used for measuring latency of blocking/non-blocking cmds */
time_t lastinteraction; /* Time of the last interaction, used for timeout */
time_t obuf_soft_limit_reached_time;
uint64_t flags; /* Client flags: CLIENT_* macros. */
int authenticated; /* Needed when the default user requires auth. */
int replstate; /* Replication state if this is a slave. */
int repl_put_online_on_ack; /* Install slave write handler on first ACK. */
int repldbfd; /* Replication DB file descriptor. */
off_t repldboff; /* Replication DB file offset. */
off_t repldbsize; /* Replication DB file size. */
sds replpreamble; /* Replication DB preamble. */
long long read_reploff; /* Read replication offset if this is a master. */
long long reploff; /* Applied replication offset if this is a master. */
long long repl_ack_off; /* Replication ack offset, if this is a slave. */
long long repl_ack_time;/* Replication ack time, if this is a slave. */
long long repl_last_partial_write; /* The last time the server did a partial write from the RDB child pipe to this replica */
long long psync_initial_offset; /* FULLRESYNC reply offset other slaves
copying this slave output buffer
should use. */
char replid[CONFIG_RUN_ID_SIZE+1]; /* Master replication ID (if master). */
int slave_listening_port; /* As configured with: REPLCONF listening-port */
char *slave_addr; /* Optionally given by REPLCONF ip-address */
int slave_capa; /* Slave capabilities: SLAVE_CAPA_* bitwise OR. */
multiState mstate; /* MULTI/EXEC state */
int btype; /* Type of blocking op if CLIENT_BLOCKED. */
blockingState bpop; /* blocking state */
long long woff; /* Last write global replication offset. */
list *watched_keys; /* Keys WATCHED for MULTI/EXEC CAS */
dict *pubsub_channels; /* channels a client is interested in (SUBSCRIBE) */
list *pubsub_patterns; /* patterns a client is interested in (SUBSCRIBE) */
sds peerid; /* Cached peer ID. */
sds sockname; /* Cached connection target address. */
listNode *client_list_node; /* list node in client list */
listNode *paused_list_node; /* list node within the pause list */
RedisModuleUserChangedFunc auth_callback; /* Module callback to execute
* when the authenticated user
* changes. */
void *auth_callback_privdata; /* Private data that is passed when the auth
* changed callback is executed. Opaque for
* Redis Core. */
void *auth_module; /* The module that owns the callback, which is used
* to disconnect the client if the module is
* unloaded for cleanup. Opaque for Redis Core.*/
/* If this client is in tracking mode and this field is non zero,
* invalidation messages for keys fetched by this client will be send to
* the specified client ID. */
uint64_t client_tracking_redirection;
rax *client_tracking_prefixes; /* A dictionary of prefixes we are already
subscribed to in BCAST mode, in the
context of client side caching. */
/* In clientsCronTrackClientsMemUsage() we track the memory usage of
* each client and add it to the sum of all the clients of a given type,
* however we need to remember what was the old contribution of each
* client, and in which categoty the client was, in order to remove it
* before adding it the new value. */
uint64_t client_cron_last_memory_usage;
int client_cron_last_memory_type;
/* Response buffer */
int bufpos;
char buf[PROTO_REPLY_CHUNK_BYTES];
} client;
套接字描述符
typedef struct client {
……
// 記錄客戶端正在使用的套接字描述符
int fd;
……
}
標誌
客戶端的標誌屬性 flags 記錄了客戶端的角色
(role),以及客戶端目前所處的狀態:
typedef struct redisClient {
// ...
int flags;
// ...
} redisClient;
具體值可參考:《Redis 設計與實現-客戶端屬性》,flag 例子:
# 客戶端是一個主服務器
REDIS_MASTER
# 客戶端正在被列表命令阻塞
REDIS_BLOCKED
# 客戶端正在執行事務,但事務的安全性已被破壞
REDIS_MULTI | REDIS_DIRTY_CAS
# 客戶端是一個從服務器,並且版本低於 Redis 2.8
REDIS_SLAVE | REDIS_PRE_PSYNC
# 這是專門用於執行 Lua 腳本包含的 Redis 命令的僞客戶端
# 它強制服務器將當前執行的命令寫入 AOF 文件,並複製給從服務器
REDIS_LUA_CLIENT | REDIS_FORCE_AOF | REDIS_FORCE_REPL
輸入緩衝區
客戶端狀態的輸入緩衝區用於保存客戶端發送的命令請求
:
typedef struct redisClient {
// ...
sds querybuf;
// ...
} redisClient;
如果客戶端向服務器發送了以下命令請求:
SET key value
客戶端狀態的 querybuf 屬性將是一個包含以下內容的 SDS 值:
*3\r\n$3\r\nSET\r\n$3\r\nkey\r\n$5\r\nvalue\r\n
展示了這個 SDS 值以及 querybuf 屬性的樣子:
命名及命令參數
在服務器將客戶端發送的命令請求保存到客戶端狀態的 querybuf
後,服務器會分析該命令,將得到的命令參數、命令參數的個數分別保存到客戶端狀態的 argv
屬性和 argc
屬性中:
typedef struct redisClient {
// ...
robj **argv;
int argc;
// ...
} redisClient;
命令的實現函數
當服務器從協議內容中分析並得出 argv 屬性和 argc 屬性的值之後, 服務器將根據項 argv[0] 的值,在命令表中查找命令所對應的命令實現函數。
當程序在命令表中成功找到 argv[0] 所對應的 redisCommand
結構時, 它會將客戶端狀態的 cmd 指針
指向這個結構:
typedef struct redisClient {
// ...
struct redisCommand *cmd;
// ...
} redisClient;
struct redisCommand {
char *name;
redisCommandProc *proc;
int arity;
char *sflags; /* Flags as string representation, one char per flag. */
uint64_t flags; /* The actual flags, obtained from the 'sflags' field. */
/* Use a function to determine keys arguments in a command line.
* Used for Redis Cluster redirect. */
redisGetKeysProc *getkeys_proc;
/* What keys should be loaded in background when calling this command? */
int firstkey; /* The first argument that's a key (0 = no keys) */
int lastkey; /* The last argument that's a key */
int keystep; /* The step between first and last key */
long long microseconds, calls, rejected_calls, failed_calls;
int id; /* Command ID. This is a progressive ID starting from 0 that
is assigned at runtime, and is used in order to check
ACLs. A connection is able to execute a given command if
the user associated to the connection has this command
bit set in the bitmap of allowed commands. */
};
每個命令所對應的處理函數在是下面的 table:
struct redisCommand redisCommandTable[] = {
{"module",moduleCommand,-2,
"admin no-script",
0,NULL,0,0,0,0,0,0},
{"get",getCommand,2,
"read-only fast @string",
0,NULL,1,1,1,0,0,0},
{"getex",getexCommand,-2,
"write fast @string",
0,NULL,1,1,1,0,0,0},
……
}
輸出緩衝區
保存執行命令所得的命令回覆
。
客戶端的固定大小緩衝區
由 buf
和 bufpos
兩個屬性組成:
typedef struct redisClient {
// ...
char buf[REDIS_REPLY_CHUNK_BYTES];
// 記錄了 buf 數組目前已使用的字節數量
int bufpos;
// ...
} redisClient;
可變大小緩衝區
由 reply 鏈表
和一個或多個字符串對象組成:
typedef struct redisClient {
// ...
list *reply;
// ...
} redisClient;
通過使用鏈表來連接多個字符串對象, 服務器可以爲客戶端保存一個非常長的命令回覆, 而不必受到固定大小緩衝區 16 KB 大小的限制。展示了一個包含三個字符串對象的 reply 鏈表。
客戶端的創建與關閉
創建普通客戶端
使用 connect 函數連接到服務器,服務器調用連接事件處理器,爲客戶端創建對應的客戶端狀態,並將其添加到服務器狀態結構 clients 鏈表的末尾。
關閉普通客戶端
關閉普通客戶端的原因:
- 客戶端進程退出或被殺死,客戶端與服務端的網絡連接被關閉
- 客戶端向服務端發送了不符合協議格式的命令請求
- 客戶端成爲了 CLIENT KILL 命令的目標
- 客戶端的空轉時間超過 timeout 配置選項的值
- 客戶端發送的命令請求大小,超過了深入緩衝區的限制大小(默認爲 1GB)
- 服務端返回給客戶端的數據超過了輸出緩衝區的限制大小
參考鏈接
Redis 源碼簡潔剖析系列
Java 編程思想-最全思維導圖-GitHub 下載鏈接,需要的小夥伴可以自取~
原創不易,希望大家轉載時請先聯繫我,並標註原文鏈接。