Redis 源碼簡潔剖析 12 - 一條命令的處理過程

命令的處理過程

Redis server 和一個客戶端建立連接後,會在事件驅動框架中註冊可讀事件——客戶端的命令請求。命令處理對應 4 個階段:

  • 命令讀取:對應 readQueryFromClient 函數
  • 命令解析:對應 processInputBuffer 函數
  • 命令執行:對應 processCommand 函數
  • 結果返回:對應 addReply 函數

命令讀取

readQueryFromClient 函數在之前的文章中分析過,主要流程就是:

  1. 調用 connRead 函數讀取命令
  2. 將命令追加到同步緩衝區,修改同步偏移量
  3. 調用 processInputBuffer 函數進行命令解析
void readQueryFromClient(connection *conn) {
    // 從 connection 結構中獲取客戶端
    client *c = connGetPrivateData(conn);
    ……
    nread = connRead(c->conn, c->querybuf+qblen, readlen);
    ……

    /* There is more data in the client input buffer, continue parsing it
     * in case to check if there is a full command to execute. */
     processInputBuffer(c);
}

命令解析

processInputBuffer 函數會調用 processCommandAndResetClient 函數,其中又會調用 processCommand 函數。

void processInputBuffer(client *c) {

    while(c->qb_pos < sdslen(c->querybuf)) {
        ……

        // 根據客戶端輸入緩衝區的命令開頭字符判斷命令類型
        if (!c->reqtype) {
            // 符合 RESP 協議的命令
            if (c->querybuf[c->qb_pos] == '*') {
                c->reqtype = PROTO_REQ_MULTIBULK;
            } else {
                // 管道類型命令
                c->reqtype = PROTO_REQ_INLINE;
            }
        }

        // 對於管道類型命令,調用 processInlineBuffer 函數解析
        if (c->reqtype == PROTO_REQ_INLINE) {
            if (processInlineBuffer(c) != C_OK) break;
            ……
        // 對於 RESP 協議命令,調用 processMultibulkBuffer 函數解析
        } else if (c->reqtype == PROTO_REQ_MULTIBULK) {
            if (processMultibulkBuffer(c) != C_OK) break;
        }
        ……

        if (c->argc == 0) {
            resetClient(c);
        } else {
            ……

            // 可以開始執行命令了
            if (processCommandAndResetClient(c) == C_ERR) {
                return;
            }
        }
    }
    ……
}
int processCommandAndResetClient(client *c) {
    int deadclient = 0;
    client *old_client = server.current_client;
    server.current_client = c;
    if (processCommand(c) == C_OK) {
        commandProcessed(c);
    }
    if (server.current_client == NULL) deadclient = 1;
    /*
     * Restore the old client, this is needed because when a script
     * times out, we will get into this code from processEventsWhileBlocked.
     * Which will cause to set the server.current_client. If not restored
     * we will return 1 to our caller which will falsely indicate the client
     * is dead and will stop reading from its buffer.
     */
    server.current_client = old_client;
    /* performEvictions may flush slave output buffers. This may
     * result in a slave, that may be the active client, to be
     * freed. */
    return deadclient ? C_ERR : C_OK;
}

命令執行

processCommand 函數是在 server.c 文件中實現的:

  • 調用 moduleCallCommandFilters 函數,將 Redis 命令替換成 module 想要替換的命令
  • 當前命令是否爲 quit 命令,並進行相應處理
  • 調用 lookupCommand 函數,在全局變量 server 的 commands 成員變量中查找相關命令

commands 是一個哈希表:

struct redisServer {
   ...
   dict *commands; 
   ...
}

其是在 initServerConfig 函數中初始化的:

void initServerConfig(void) {
    ...
    server.commands = dictCreate(&commandTableDictType,NULL);
    ...
    populateCommandTable();
    ...
}

populateCommandTable 函數中使用了 redisCommandTable 數組:

void populateCommandTable(void) {
    int j;
    int numcommands = sizeof(redisCommandTable)/sizeof(struct redisCommand);

    for (j = 0; j < numcommands; j++) {
        struct redisCommand *c = redisCommandTable+j;
        int retval1, retval2;

        /* Translate the command string flags description into an actual
         * set of flags. */
        if (populateCommandTableParseFlags(c,c->sflags) == C_ERR)
            serverPanic("Unsupported command flag");

        c->id = ACLGetCommandID(c->name); /* Assign the ID used for ACL. */
        retval1 = dictAdd(server.commands, sdsnew(c->name), c);
        /* Populate an additional dictionary that will be unaffected
         * by rename-command statements in redis.conf. */
        retval2 = dictAdd(server.orig_commands, sdsnew(c->name), c);
        serverAssert(retval1 == DICT_OK && retval2 == DICT_OK);
    }
}

redisCommandTable 數組是在 server.c 中定義的,記錄了當前命令所對應的實現函數。具體見:https://github.com/LjyYano/redis/blob/unstable/src/server.c

struct redisCommand redisCommandTable[] = {
    {"module",moduleCommand,-2,
     "admin no-script",
     0,NULL,0,0,0,0,0,0},

    {"get",getCommand,2,
     "read-only fast @string",
     0,NULL,1,1,1,0,0,0},

    {"getex",getexCommand,-2,
     "write fast @string",
     0,NULL,1,1,1,0,0,0},

     ……
};

redisCommand 結構如下:

struct redisCommand {
    char *name;
    redisCommandProc *proc;
    int arity;
    char *sflags;   /* Flags as string representation, one char per flag. */
    uint64_t flags; /* The actual flags, obtained from the 'sflags' field. */
    /* Use a function to determine keys arguments in a command line.
     * Used for Redis Cluster redirect. */
    redisGetKeysProc *getkeys_proc;
    /* What keys should be loaded in background when calling this command? */
    int firstkey; /* The first argument that's a key (0 = no keys) */
    int lastkey;  /* The last argument that's a key */
    int keystep;  /* The step between first and last key */
    long long microseconds, calls, rejected_calls, failed_calls;
    int id;     /* Command ID. This is a progressive ID starting from 0 that
                   is assigned at runtime, and is used in order to check
                   ACLs. A connection is able to execute a given command if
                   the user associated to the connection has this command
                   bit set in the bitmap of allowed commands. */
};

再回到 processCommand 函數,斷當前客戶端是否有 CLIENT_MULTI 標記,如果有的話,就表明要處理的是 Redis 事務的相關命令,所以它會按照事務的要求,調用 queueMultiCommand 函數將命令入隊保存,等待後續一起處理。而如果沒有,processCommand 函數就會調用 call 函數來實際執行命令了。

if (c->flags & CLIENT_MULTI &&
    c->cmd->proc != execCommand && c->cmd->proc != discardCommand &&
    c->cmd->proc != multiCommand && c->cmd->proc != watchCommand &&
    c->cmd->proc != resetCommand)
{
    // 將命令入隊保存,後續一起處理
    queueMultiCommand(c);
    addReply(c,shared.queued);
} else {
    // 調用 call 函數執行命令
    call(c,CMD_CALL_FULL);
    ……
}

下面以最簡單的 get 命令爲例:

{"get",getCommand,2,
    "read-only fast @string",
    0,NULL,1,1,1,0,0,0},

對應的實現函數是 getCommand,其調用了 getGenericCommand 函數:

void getCommand(client *c) {
    getGenericCommand(c);
}

int getGenericCommand(client *c) {
    robj *o;

    if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.null[c->resp])) == NULL)
        return C_OK;

    if (checkType(c,o,OBJ_STRING)) {
        return C_ERR;
    }

    addReplyBulk(c,o);
    return C_OK;
}

其最終會調用到 db.c 文件中的 lookupKeyReadWithFlags 函數:

robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags) {
    robj *val;

    if (expireIfNeeded(db,key) == 1) {
        /* If we are in the context of a master, expireIfNeeded() returns 1
         * when the key is no longer valid, so we can return NULL ASAP. */
        if (server.masterhost == NULL)
            goto keymiss;

        /* However if we are in the context of a slave, expireIfNeeded() will
         * not really try to expire the key, it only returns information
         * about the "logical" status of the key: key expiring is up to the
         * master in order to have a consistent view of master's data set.
         *
         * However, if the command caller is not the master, and as additional
         * safety measure, the command invoked is a read-only command, we can
         * safely return NULL here, and provide a more consistent behavior
         * to clients accessing expired values in a read-only fashion, that
         * will say the key as non existing.
         *
         * Notably this covers GETs when slaves are used to scale reads. */
        if (server.current_client &&
            server.current_client != server.master &&
            server.current_client->cmd &&
            server.current_client->cmd->flags & CMD_READONLY)
        {
            goto keymiss;
        }
    }
    val = lookupKey(db,key,flags);
    if (val == NULL)
        goto keymiss;
    server.stat_keyspace_hits++;
    return val;

keymiss:
    if (!(flags & LOOKUP_NONOTIFY)) {
        notifyKeyspaceEvent(NOTIFY_KEY_MISS, "keymiss", key, db->id);
    }
    server.stat_keyspace_misses++;
    return NULL;
}

會調用到 lookupKey 函數:

robj *lookupKey(redisDb *db, robj *key, int flags) {
    dictEntry *de = dictFind(db->dict,key->ptr);
    if (de) {
        robj *val = dictGetVal(de);

        /* Update the access time for the ageing algorithm.
         * Don't do it if we have a saving child, as this will trigger
         * a copy on write madness. */
        if (!hasActiveChildProcess() && !(flags & LOOKUP_NOTOUCH)){
            if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
                updateLFU(val);
            } else {
                val->lru = LRU_CLOCK();
            }
        }
        return val;
    } else {
        return NULL;
    }
}

結果返回

addReply 函數,主要是調用 prepareClientToWrite 函數,進而調用到 clientInstallWriteHandler 函數,將待寫回客戶端加入到全局變量 server 的 clients_pending_write 列表。最終調用 _addReplyToBuffer 函數,將要返回的結果添加到客戶端的輸出緩衝區。

/* Add the object 'obj' string representation to the client output buffer. */
void addReply(client *c, robj *obj) {
    if (prepareClientToWrite(c) != C_OK) return;

    if (sdsEncodedObject(obj)) {
        if (_addReplyToBuffer(c,obj->ptr,sdslen(obj->ptr)) != C_OK)
            _addReplyProtoToList(c,obj->ptr,sdslen(obj->ptr));
    } else if (obj->encoding == OBJ_ENCODING_INT) {
        /* For integer encoded strings we just convert it into a string
         * using our optimized function, and attach the resulting string
         * to the output buffer. */
        char buf[32];
        size_t len = ll2string(buf,sizeof(buf),(long)obj->ptr);
        if (_addReplyToBuffer(c,buf,len) != C_OK)
            _addReplyProtoToList(c,buf,len);
    } else {
        serverPanic("Wrong obj->encoding in addReply()");
    }
}

參考鏈接

Redis 源碼簡潔剖析系列

最簡潔的 Redis 源碼剖析系列文章

Java 編程思想-最全思維導圖-GitHub 下載鏈接,需要的小夥伴可以自取~

原創不易,希望大家轉載時請先聯繫我,並標註原文鏈接。

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章