Queue data structures are common and useful, but not always thread-safe. Jeff Cogswell explains how a queue works and why a typical implementation won’t cut it for multithreaded programming. He then introduces the Threading Building Blocks queue structure, which scales nicely for high-performance, multithreaded programming.
A common and useful data structure in computer science is a queue. In general English usage, the word “queue” refers to a line of people, where the person in front goes first (such as at a bank or a grocery check-out). As people arrive, they go to the back of the line. The term has pretty much the same meaning in computer science, whereby a queue is a list where you take one item off the front of the line, and as a more items come in, you add them to the back of the line.
Most of today’s programming languages include a queue data structure. C++ is an older language, and although the original C++ language didn’t have a queue, it was easy to build one with the existing features. Starting with the C++ standard of 1998, a queue was included in the official standard, although it wasn’t an atomic type built into the language; rather, the standard included a queue structure built on top of existing types.
However, the problem with the standard queue is that it isn’t necessarily thread-safe. A queue has two basic operations: A “push” operation, which adds an item to the back of the queue, and a “pop” item, which retrieves the front item off of the queue for use in some sort of data processing. To understand why it’s easy to create a queue structure that isn’t thread-safe, think about how you would implement those two operations. There are different ways to do it, but here’s one way, using a doubly-linked list.
I’m going to be brief here and not give all the details, just the important parts to make the point. But the queue would maintain a pointer to the first-in-line item and the last-in-line item. The push operation would require taking the address of the last-in-line item, storing this address inside the item being added, and storing this item’s address in what was the last-in-line item. Then the queue would point to this new item. The pop item would mean retrieving the address of the final item, and then having the queue point to the preceding item for the last-in-line item.
Now look at the problem we’re encountering if we were to create a queue in this manner and try to use it in multiple threads. One problem occurs if two threads try to simultaneously add items to the end of the queue. The first thread might take the address of the current last-in-line item, and before that thread can insert its item, the other thread might also take the address of the current last-in-line item. A race condition occurs, because then it’s just a matter of which thread moves faster. You may end up with one thread not actually inserting its item, or you might end up with one item pointing to what was previously the last-in-line, and with the previous last-in-line pointing to the other item. That would be a total mess.
One way around this is to treat the entire push operation as atomic, and the entire pop operation as atomic. There are different ways to do this, including using some features built into the Intel assembly language, as well as using critical sections of code. However, when you’re using critical sections, you want to make them fast and efficient so that they get their job done as quickly as possible so as to not slow the whole thing down and defeat the whole purpose of using parallelization.
That’s where Threading Building Blocks (TBB) comes in. The TBB threading library includes a queue structure that is carefully-crafted to take advantage of the processor and language features such that the operations are atomic, and with minimal impact on performance. And the great thing is the queue is incredibly easy to use. Normally when I write examples, I have lots of code and have to explain it. But the beautiful thing here is there’s really not a whole lot to explain. The class is called concurrent_queue and, like most TBB classes, it’s a template. Here’s an example line of creating the queue:
tbb::concurrent_queue<int> my_queue;
That creates a queue that holds integers, and stores the queue in the my_queue variable.
Then to push items into the queue, you just call push, regardless of the thread you’re in:
my_queue.push(10);
To pop an item off the queue, call try_pop. This function takes as a parameter a reference, which means you pass in the variable you want to receive the contents of the item in the front of the queue. The function returns a boolean telling you if the pop was successful. Why wouldn’t it be successful? If the queue is empty it would fail:
if (my_queue.try_pop(num)) {
. . .
}
There’s also an iterator, which might seem a bit odd for a queue, but it’s certainly useful for pulling out all the items in the queue one by one. Take a look at the docs for a complete example.
Conclusion
The queue is an incredibly simple yet useful data structure in computer programming. But a lot of queue implementations were written without thread-safety in mind. Threading Building Blocks, however, includes a queue structure that easily scales for parallel processing.