手撕環形隊列系列二：無鎖實現高併發

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文是手撕環形隊列系列的第二篇，之前的文章鏈接如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://xie.infoq.cn/article/bc86e0b32a5f6c1c7888d631f","title":"","type":null},"content":[{"type":"text","text":"《手撕環形隊列》","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面文章介紹的是一個比較基本的環形隊列，能夠在多線程中使用，但有一個前提：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"任意時刻，生產者和消費者最多都只能有一個。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"也就是說，如果有多個生產者要併發向隊列中寫入，需要在外部進行加鎖或其它方式的併發控制，保證任意時刻最多隻有一個生產者真正向環形隊列進行寫入。同樣的，多個消費者要從隊列中讀取進行消費，也需要在外部進行加鎖或其它方式的併發控制，保證任意時刻最多隻有一個消費者從環形隊列進行讀取。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"本文的內容，就是介紹如何能夠支持多線程場景下，多生產者併發寫入、多消費者併發讀取，完全由環形隊列內部來解決，無需外部做任何額外的控制。並且，使用無鎖的技術來實現，從而避免加鎖解鎖這種重操作對性能的影響。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"無鎖數據結構中，主要的技術實現手段是使用cpu的","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"原子指令","attrs":{}},{"type":"text","text":"。介紹原子指令之前，先介紹一下沒有原子指令的情況下會有什麼問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通常我們在程序源碼中寫的語句，編譯爲二進制後，代碼中的一行文本語句會變成二進制的多條彙編指令，因此這一行文本語句cpu執行時就不是原子的。多行文本語句，就更不是原子的了。多線程併發執行這些文本語句時，對應的多行彙編語句會在多個cpu 核上同時執行，無法保證他們之間的執行先後順序關係。在多線程同時讀寫一個共享數據時，會發生各種誤判，導致錯誤的結果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以環形隊列爲例，來說明這個問題：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"環形隊列爲初始狀態，隊列爲空。兩個生產者線程都要向隊列進行寫入，都調用 ring_queue_push()方法。這個方法的函數實現中，producer1 線程讀取tail 爲0，producer2 線程也讀取到tail爲0。然後producer1 向位置0寫入數據，然後把tail 增1，tail變爲1。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"producer2 也向位置0寫入數據，然後把tail 增1. tai增加1的過程：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"tail = tail + 1; ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於producer2 初始讀取的tail 值爲0，這個cpu core 可能意識不到tail 已經被別的線程修改了，因此還認爲tail是0，因此最終","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"tail = 0 + 1 = 1;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"最終的結果，producer2 把producer1的數據給覆蓋了(數據丟了)，但兩個ring_queue_push()函數調用都返回成功了。這是一個嚴重的Bug！","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實際多線程環境中，各個cpu 之間的代碼執行時序都是不同的，因此沒有任何防護的情況下，對同樣的內存位置寫入、對同一個變量的併發讀和併發寫，都會產生嚴重的Bug。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了解決這些問題，原子指令閃亮登場了！","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用這些指令，對數據的操作在多cpu的情況下也是原子性的。所謂原子性，就是作爲執行的最小單位，不能再分割。cpu core 要麼執行了這個指令，要麼還沒執行這個指令。不會出現在一個cpu core 執行這個指令一半的時候，另外一個cpu core開始執行這個指令的情況。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過正確使用cpu的原子指令，能夠有效解決多線程併發中的各種問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在解決多線程併發問題，常規的方法是用mutex、semaphore、condvar等，這些可以理解爲粗粒度鎖，使用簡單，適用範圍廣，但性能較差。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"cpu的原子指令，是cpu指令級的細粒度鎖，性能非常高，但設計起來複雜。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"各種操作系統、開發語言中都提供了對cpu原子指令的包裝函數，因此不需要我們手寫彙編指令。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以gcc爲例，gcc提供了 一系列builtin 的原子函數，比如今天我們要用的：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"bool __sync_bool_compare_and_swap(type *ptr, type oldval， type newval);","attrs":{}}],"attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這個函數，會將 ptr指向內存中的值，與oldval 比較，如果相等，則把 ptr執行內存的值修改爲 newval. 整個比較和修改的全過程，要以原子方式完成。如果比較相等，並且修改成功，則返回true。其它情況都返回false。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這個函數，也叫 cas，取的是 compare and swap 三個單詞的首字母縮寫。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們用原子指令，來增強一下環形隊列，實現多生產多消費者併發讀寫。思路如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"對於寫入，每個producer 必須先獲得寫鎖。成功獲得寫鎖之後，寫入數據，將tail移動到下一個位置，最後釋放寫鎖。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"對於讀取，每個consumer 必須先獲得讀鎖。成功獲得讀鎖之後，讀取數據，將head移動到下一個位置，最後釋放讀鎖。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整個思路，與傳統通過mutex控制對共享數據的讀寫是完全一樣的，只是技術實現上我們用原子指令來實現，這種實現方式叫無鎖數據結構。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外，需要說明的是：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於head和tail這樣的變量，由於多個線程會併發讀寫，因此我們需要用 volatile 來修飾它們，不讓cpu core 緩存它們，避免讀到舊數據。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"無鎖環形隊列，支持多生產者多消費者併發讀寫，用C語言實現的源碼如下：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":"c"},"content":[{"type":"text","text":"// ring_queue.h\n#ifndef RING_QUEUE_H\n#define RING_QUEUE_H\n\ntypedef struct ring_queue_t {\n char* pbuf;\n int item_size;\n int capacity;\n\n volatile int write_flag;\n volatile int read_flag;\n\n volatile int head;\n volatile int tail;\n volatile int same_cycle;\n} ring_queue_t;\n\nint ring_queue_init(ring_queue_t* pqueue, int item_size, int capacity);\nvoid ring_queue_destroy(ring_queue_t* pqueue);\nint ring_queue_push(ring_queue_t* pqueue, void* pitem);\nint ring_queue_pop(ring_queue_t* pqueue, void* pitem);\nint ring_queue_is_empty(ring_queue_t* pqueue);\nint ring_queue_is_full(ring_queue_t* pqueue);\n\n#endif\n\n","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"c"},"content":[{"type":"text","text":"// ring_queue.c\n#include \"ring_queue.h\"\n#include \n#include \n\n#define CAS(ptr, old, new) __sync_bool_compare_and_swap(ptr, old, new)\n\nint ring_queue_init(ring_queue_t* pqueue, int item_size, int capacity) {\n memset(pqueue, 0, sizeof(*pqueue));\n pqueue->pbuf = (char*)malloc(item_size * capacity);\n if (!pqueue->pbuf) {\n return -1;\n }\n\n pqueue->item_size = item_size;\n pqueue->capacity = capacity;\n pqueue->same_cycle = 1;\n return 0;\n}\n\nvoid ring_queue_destroy(ring_queue_t* pqueue) {\n free(pqueue->pbuf);\n memset(pqueue, 0, sizeof(*pqueue));\n}\n\nint ring_queue_push(ring_queue_t* pqueue, void* pitem) {\n // try to set write flag\n while (1) {\n if (ring_queue_is_full(pqueue)) {\n return -1;\n }\n\n if (CAS(&pqueue->write_flag, 0, 1)) { // set write flag successfully\n break;\n }\n }\n\n // push data\n memcpy(pqueue->pbuf + pqueue->tail * pqueue->item_size, pitem, pqueue->item_size);\n pqueue->tail = (pqueue->tail + 1) % pqueue->capacity;\n if (0 == pqueue->tail) { // a new cycle\n pqueue->same_cycle = 0; // tail is not the same cycle with head\n }\n\n // reset write flag\n CAS(&pqueue->write_flag, 1, 0);\n\n return 0;\n}\n\nint ring_queue_pop(ring_queue_t* pqueue, void* pitem) {\n // try to set read flag\n while (1) {\n if (ring_queue_is_empty(pqueue)) {\n return -1;\n }\n\n if (CAS(&pqueue->read_flag, 0, 1)) { // set read flag successfully\n break;\n }\n }\n\n // read data\n memcpy(pitem, pqueue->pbuf + pqueue->head * pqueue->item_size, pqueue->item_size);\n pqueue->head = (pqueue->head + 1) % pqueue->capacity;\n if (0 == pqueue->head) {\n pqueue->same_cycle = 1; // head is now the same cycle with tail\n }\n\n // reset read flag\n CAS(&pqueue->read_flag, 1, 0);\n\n return 0;\n}\n\nint ring_queue_is_empty(ring_queue_t* pqueue) {\n return (pqueue->head == pqueue->tail) && pqueue->same_cycle;\n}\n\nint ring_queue_is_full(ring_queue_t* pqueue) {\n return (pqueue->head == pqueue->tail) && !pqueue->same_cycle;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我的微信號是 實力程序員，歡迎大家轉發至朋友圈，分享給更多的朋友。","attrs":{}}]}]}