讀懂纔會用 ： Redis的多線程

2020.5.2日，Redis 6.0.1正式發佈。除了增加新功能和新的API，支持多線程是最大變化。本文以簡化的方式理解Redis線程模型的演進。

Redis單線程

嚴格講，Redis 並不是單線程。有後臺線程在工作，處理一些較爲緩慢的操作，例如無用連接的釋放、大key的刪除等。client端命令的請求獲取 (socket 讀)、解析、執行、內容返回 (socket 寫) 都是由一個線程處理，因此我們常說的“單線程”指的是處理核心處理的線程只有一個。

處理流程

如下圖：

Redis採用evport，epoll，kqueue和select四種方式實現多路複用，提升鏈接處理，Linux平臺默認是epoll

參考ae.c文件L47

/* Include the best multiplexing layer supported by this system.
 * The following should be ordered by performances, descending. */
#ifdef HAVE_EVPORT
#include "ae_evport.c"
#else
    #ifdef HAVE_EPOLL
    #include "ae_epoll.c"
    #else
        #ifdef HAVE_KQUEUE
        #include "ae_kqueue.c"
        #else
        #include "ae_select.c"
        #endif
    #endif
#endif

優點 VS 缺點

優勢：

1. 不存在鎖的問題
2. 避免線程間CPU切換

缺點：

1. 單線程無法利用多CPU
2. 串行操作，某個操作“出問題”會“阻塞”後續操作

Redis 6 多線程

處理流程

默認並不開啓多線程，需要參數設置，IO_THREADS_MAX_NUM 最大爲128

network.c文件 L2886

/* Initialize the data structures needed for threaded I/O. */
void initThreadedIO(void) {
    io_threads_active = 0; /* We start with threads not active. */

    /* Don't spawn any thread if the user selected a single thread:
     * we'll handle I/O directly from the main thread. */
    if (server.io_threads_num == 1) return;

    if (server.io_threads_num > IO_THREADS_MAX_NUM) {
        serverLog(LL_WARNING,"Fatal: too many I/O threads configured. "
                             "The maximum number is %d.", IO_THREADS_MAX_NUM);
        exit(1);
    }

    /* Spawn and initialize the I/O threads. */
    for (int i = 0; i < server.io_threads_num; i++) {
        /* Things we do for all the threads including the main thread. */
        io_threads_list[i] = listCreate();
        if (i == 0) continue; /* Thread 0 is the main thread. */

        /* Things we do only for the additional threads. */
        pthread_t tid;
        pthread_mutex_init(&io_threads_mutex[i],NULL);
        io_threads_pending[i] = 0;
        pthread_mutex_lock(&io_threads_mutex[i]); /* Thread will be stopped. */
        if (pthread_create(&tid,NULL,IOThreadMain,(void*)(long)i) != 0) {
            serverLog(LL_WARNING,"Fatal: Can't initialize IO thread.");
            exit(1);
        }
        io_threads[i] = tid;
    }
}

加入隊列

network.c文件L3040

/* Return 1 if we want to handle the client read later using threaded I/O.
 * This is called by the readable handler of the event loop.
 * As a side effect of calling this function the client is put in the
 * pending read clients and flagged as such. */
int postponeClientRead(client *c) {
    if (io_threads_active &&
        server.io_threads_do_reads &&
        !ProcessingEventsWhileBlocked &&
        !(c->flags & (CLIENT_MASTER|CLIENT_SLAVE|CLIENT_PENDING_READ)))
    {
        c->flags |= CLIENT_PENDING_READ;
        listAddNodeHead(server.clients_pending_read,c);
        return 1;
    } else {
        return 0;
    }
}

/* When threaded I/O is also enabled for the reading + parsing side, the
 * readable handler will just put normal clients into a queue of clients to
 * process (instead of serving them synchronously). This function runs
 * the queue using the I/O threads, and process them in order to accumulate
 * the reads in the buffers, and also parse the first command available
 * rendering it in the client structures. */
int handleClientsWithPendingReadsUsingThreads(void) {

基礎類圖

優點 VS 缺點

優點：

1、提高響應速度，充分使用CPU

缺點：

1、增加了代碼複雜性

總結

1. Redis的多路複用技術，支持epoll、kqueue、selector
2. 5.0版本及以前，處理客戶端請求的線程只有一個，串行處理
3. 6.0版本引入了worker Thread，只處理網絡IO讀取和寫入，核心IO負責串行處理客戶端指令

系列

讀懂纔會用 ： 瞅瞅Redis的epoll模型
讀懂纔會用 ： Redis的多線程

關注我

如果您在微信閱讀，請您點擊鏈接 關注我 ，如果您在 PC 上閱讀請掃碼關注我，歡迎與我交流隨時指出錯誤