手撕环形队列系列二：无锁实现高并发

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文是手撕环形队列系列的第二篇，之前的文章链接如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://xie.infoq.cn/article/bc86e0b32a5f6c1c7888d631f","title":"","type":null},"content":[{"type":"text","text":"《手撕环形队列》","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面文章介绍的是一个比较基本的环形队列，能够在多线程中使用，但有一个前提：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"任意时刻，生产者和消费者最多都只能有一个。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"也就是说，如果有多个生产者要并发向队列中写入，需要在外部进行加锁或其它方式的并发控制，保证任意时刻最多只有一个生产者真正向环形队列进行写入。同样的，多个消费者要从队列中读取进行消费，也需要在外部进行加锁或其它方式的并发控制，保证任意时刻最多只有一个消费者从环形队列进行读取。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"本文的内容，就是介绍如何能够支持多线程场景下，多生产者并发写入、多消费者并发读取，完全由环形队列内部来解决，无需外部做任何额外的控制。并且，使用无锁的技术来实现，从而避免加锁解锁这种重操作对性能的影响。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"无锁数据结构中，主要的技术实现手段是使用cpu的","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"原子指令","attrs":{}},{"type":"text","text":"。介绍原子指令之前，先介绍一下没有原子指令的情况下会有什么问题。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通常我们在程序源码中写的语句，编译为二进制后，代码中的一行文本语句会变成二进制的多条汇编指令，因此这一行文本语句cpu执行时就不是原子的。多行文本语句，就更不是原子的了。多线程并发执行这些文本语句时，对应的多行汇编语句会在多个cpu 核上同时执行，无法保证他们之间的执行先后顺序关系。在多线程同时读写一个共享数据时，会发生各种误判，导致错误的结果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以环形队列为例，来说明这个问题：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"环形队列为初始状态，队列为空。两个生产者线程都要向队列进行写入，都调用 ring_queue_push()方法。这个方法的函数实现中，producer1 线程读取tail 为0，producer2 线程也读取到tail为0。然后producer1 向位置0写入数据，然后把tail 增1，tail变为1。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"producer2 也向位置0写入数据，然后把tail 增1. tai增加1的过程：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"tail = tail + 1; ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由于producer2 初始读取的tail 值为0，这个cpu core 可能意识不到tail 已经被别的线程修改了，因此还认为tail是0，因此最终","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"tail = 0 + 1 = 1;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"最终的结果，producer2 把producer1的数据给覆盖了(数据丢了)，但两个ring_queue_push()函数调用都返回成功了。这是一个严重的Bug！","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"实际多线程环境中，各个cpu 之间的代码执行时序都是不同的，因此没有任何防护的情况下，对同样的内存位置写入、对同一个变量的并发读和并发写，都会产生严重的Bug。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为了解决这些问题，原子指令闪亮登场了！","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用这些指令，对数据的操作在多cpu的情况下也是原子性的。所谓原子性，就是作为执行的最小单位，不能再分割。cpu core 要么执行了这个指令，要么还没执行这个指令。不会出现在一个cpu core 执行这个指令一半的时候，另外一个cpu core开始执行这个指令的情况。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通过正确使用cpu的原子指令，能够有效解决多线程并发中的各种问题。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在解决多线程并发问题，常规的方法是用mutex、semaphore、condvar等，这些可以理解为粗粒度锁，使用简单，适用范围广，但性能较差。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"cpu的原子指令，是cpu指令级的细粒度锁，性能非常高，但设计起来复杂。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"各种操作系统、开发语言中都提供了对cpu原子指令的包装函数，因此不需要我们手写汇编指令。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以gcc为例，gcc提供了 一系列builtin 的原子函数，比如今天我们要用的：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"bool __sync_bool_compare_and_swap(type *ptr, type oldval， type newval);","attrs":{}}],"attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这个函数，会将 ptr指向内存中的值，与oldval 比较，如果相等，则把 ptr执行内存的值修改为 newval. 整个比较和修改的全过程，要以原子方式完成。如果比较相等，并且修改成功，则返回true。其它情况都返回false。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这个函数，也叫 cas，取的是 compare and swap 三个单词的首字母缩写。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们用原子指令，来增强一下环形队列，实现多生产多消费者并发读写。思路如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"对于写入，每个producer 必须先获得写锁。成功获得写锁之后，写入数据，将tail移动到下一个位置，最后释放写锁。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"对于读取，每个consumer 必须先获得读锁。成功获得读锁之后，读取数据，将head移动到下一个位置，最后释放读锁。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整个思路，与传统通过mutex控制对共享数据的读写是完全一样的，只是技术实现上我们用原子指令来实现，这种实现方式叫无锁数据结构。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外，需要说明的是：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于head和tail这样的变量，由于多个线程会并发读写，因此我们需要用 volatile 来修饰它们，不让cpu core 缓存它们，避免读到旧数据。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"无锁环形队列，支持多生产者多消费者并发读写，用C语言实现的源码如下：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":"c"},"content":[{"type":"text","text":"// ring_queue.h\n#ifndef RING_QUEUE_H\n#define RING_QUEUE_H\n\ntypedef struct ring_queue_t {\n char* pbuf;\n int item_size;\n int capacity;\n\n volatile int write_flag;\n volatile int read_flag;\n\n volatile int head;\n volatile int tail;\n volatile int same_cycle;\n} ring_queue_t;\n\nint ring_queue_init(ring_queue_t* pqueue, int item_size, int capacity);\nvoid ring_queue_destroy(ring_queue_t* pqueue);\nint ring_queue_push(ring_queue_t* pqueue, void* pitem);\nint ring_queue_pop(ring_queue_t* pqueue, void* pitem);\nint ring_queue_is_empty(ring_queue_t* pqueue);\nint ring_queue_is_full(ring_queue_t* pqueue);\n\n#endif\n\n","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"c"},"content":[{"type":"text","text":"// ring_queue.c\n#include \"ring_queue.h\"\n#include \n#include \n\n#define CAS(ptr, old, new) __sync_bool_compare_and_swap(ptr, old, new)\n\nint ring_queue_init(ring_queue_t* pqueue, int item_size, int capacity) {\n memset(pqueue, 0, sizeof(*pqueue));\n pqueue->pbuf = (char*)malloc(item_size * capacity);\n if (!pqueue->pbuf) {\n return -1;\n }\n\n pqueue->item_size = item_size;\n pqueue->capacity = capacity;\n pqueue->same_cycle = 1;\n return 0;\n}\n\nvoid ring_queue_destroy(ring_queue_t* pqueue) {\n free(pqueue->pbuf);\n memset(pqueue, 0, sizeof(*pqueue));\n}\n\nint ring_queue_push(ring_queue_t* pqueue, void* pitem) {\n // try to set write flag\n while (1) {\n if (ring_queue_is_full(pqueue)) {\n return -1;\n }\n\n if (CAS(&pqueue->write_flag, 0, 1)) { // set write flag successfully\n break;\n }\n }\n\n // push data\n memcpy(pqueue->pbuf + pqueue->tail * pqueue->item_size, pitem, pqueue->item_size);\n pqueue->tail = (pqueue->tail + 1) % pqueue->capacity;\n if (0 == pqueue->tail) { // a new cycle\n pqueue->same_cycle = 0; // tail is not the same cycle with head\n }\n\n // reset write flag\n CAS(&pqueue->write_flag, 1, 0);\n\n return 0;\n}\n\nint ring_queue_pop(ring_queue_t* pqueue, void* pitem) {\n // try to set read flag\n while (1) {\n if (ring_queue_is_empty(pqueue)) {\n return -1;\n }\n\n if (CAS(&pqueue->read_flag, 0, 1)) { // set read flag successfully\n break;\n }\n }\n\n // read data\n memcpy(pitem, pqueue->pbuf + pqueue->head * pqueue->item_size, pqueue->item_size);\n pqueue->head = (pqueue->head + 1) % pqueue->capacity;\n if (0 == pqueue->head) {\n pqueue->same_cycle = 1; // head is now the same cycle with tail\n }\n\n // reset read flag\n CAS(&pqueue->read_flag, 1, 0);\n\n return 0;\n}\n\nint ring_queue_is_empty(ring_queue_t* pqueue) {\n return (pqueue->head == pqueue->tail) && pqueue->same_cycle;\n}\n\nint ring_queue_is_full(ring_queue_t* pqueue) {\n return (pqueue->head == pqueue->tail) && !pqueue->same_cycle;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我的微信号是 实力程序员，欢迎大家转发至朋友圈，分享给更多的朋友。","attrs":{}}]}]}