tags:

views:

2690

answers:

11

I ve been googling quite a bit for a lock-free queue in C++. I found some code and some trials - but nothing that i was able to compile. A lock-free hash would also be welcome.

SUMMARY: So far i have no positive answer. There is no "production ready" library, and amazingly none of the existent libraries complies to the API of STL containers.

+8  A: 

The starting point would be either of Herb Sutter's DDJ articles for either a single producer and consumer or multiple ones. The code he gives (in-line starting on the second page of each article) uses the C++0x style atomic<T> template type; which you can imitate using the Boost interprocess library.

The boost code is buried in the depths of the interprocess library, but having read through the appropriate header file (atomic.hpp) the implementations for the necessary compare-and-swap operations on the systems I am familiar with look sound.

Steve Gilham
Steve, I'm also interested in Boost's atomic implementations, but they seem to reside in Interprocess' detail/ and aren't documented. Are they safe to use anyhow? Thanks!
Kim Gräsman
I know Herb Sutter's articles very well - where have you found the sources? They are not published by DDJ nor on his site (or maybe i am blind?).
RED SOFT ADAIR
The code is in-line in those articles starting on their respective second pages.
Steve Gilham
"Imitating" a atomic template is not what i meant with production ready. Obviously thats why Herb did not publish a source file. Did you (or anybody else) extract a working implementation from the article?
RED SOFT ADAIR
Here's a transcription of Sutter's single-producer/single-consumer queue -- http://tinesware.blogspot.com/2009/07/simple-lockfree-queue-c-windows.html
Steve Gilham
This one seems to work, but single-producer/single-consumer does not the job. What is needed are at least multiple producers.
RED SOFT ADAIR
If you want truly lock-free code, for multiple priducers or consumers, then I'm out. Sutter's multiple-producer queue example isn't lock free -- there is a lock to serialize producers, and a lock to serialize consumers. If you can find one, I'd be interested in that as well.
Steve Gilham
There is a boost::lockfree project at http://tim.klingt.org/git?p=boost_lockfree.git; that you can look at. One of its goals is to provide a non ::details:: version of atomic primitives.
sstock
+1  A: 

To the best of my knowledge, there is no such thing publicly available yet. One issue an implementor needs to solve is that you need a lock-free memory allocator, which exists, though I cannot seem to find the link right now.

Tobias
+2  A: 

And then Intel Threading Building Blocks came. And for a time, it was good.

PS : you are looking for concurrent_queue and concurrent_hash_map

Edouard A.
Those are not lock-free. See http://software.intel.com/en-us/forums/intel-threading-building-blocks/topic/55541/
RED SOFT ADAIR
I know, to the strict sens of lock-free, but I nevertheless thought it might help the OP with his problem, as the lock-free thing is just an implementation detail. I thought he was looking for collections that work well with concurrent access.
Edouard A.
+1  A: 

Hope this help :
STXXL library
TSTL - Thread Safe Template Library

lsalamon
The STXXL is not lockfree - It is for huge datasets.TSTL would be the thing, but it does not seem to be stable so far. It crashes with various memory corruptions with my project.
RED SOFT ADAIR
TSTL have new version, try again.
lsalamon
+5  A: 

After having checked most of the given answers, i can only state:

The answer is NO.

There is no such thing right that could be used right out of the box.

RED SOFT ADAIR
100% correct. I came to the same result with help from comp.programming.threads newsgroup. One reason is that the area of lock free data structures is a patent mine field. So even commercial libs like Intels are avoiding it.
Lothar
http://www.liblfds.org
Blank Xavier
This is C, not C++. Please read questions before down voting.
RED SOFT ADAIR
Apologies. I note SO will not let me undo my vote as it feels the vote is too old. I think the SO developers need more to do - they seem to be adding increasing numbers of unhelpful behaviours.
Blank Xavier
+2  A: 

boost.lockfree is an attempt to create c++ implementations of lockfree stack and fifo classes.

public git repository

Documentation: http://tim.klingt.org/boost_lockfree/
+1  A: 

If you have a multiple-producer / single-consumer Queue/FIFO, you can easily make one LockFree using SLIST or a trivial Lock Free LIFO stack. What you do is have a second "private" stack for the consumer (which can also be done as a SLIST for simplicity or any other stack model you choose). The consumer pops items off the private stack. Whenever the private LIFO is exhasted, you do a Flush rather than Pop off the shared concurrent SLIST (grabbing the entire SLIST chain) and then walk the Flushed list in-order pushing items onto the private stack.

That works for single-producer / single-consumer and for multiple-producer / single-consumer.

However, it does not work for multiple-producer / multiple-consumer cases.

Also, as far as hash tables go, they are an ideal candidate for "striping" which is just dividing the hash into segments having a lock per segments of the cache. This is how the Java concurrent library does it (using 32-stripes). If you have a light-weight reader-writer lock, the hash table can be concurrently accessed for simultaneous reads and you will only stall when write are occurring on contested stripes (and possibly if you allow for growing the hash table).

If you roll your own, make sure to interleave your locks with the hash entries rather than put all your locks in an array next to each other so you're less likely to have false sharing.

Adisak
Thanks for your Answer. I am looking for a "production ready" solution / template in C++. I dont want to roll my own. Do you know such a implementation?
RED SOFT ADAIR
+4  A: 

The closest thing I am aware of is Windows Interlocked Singly Linked Lists. Of course, it is Windows only.

Nemanja Trifunovic
Wow - that seems to be it. I'll need some time to check it (i cant do it currently) out but i'll return to you.
RED SOFT ADAIR
+2  A: 

There is such a library, but it's in C.

Wrapping to C++ should be straightforward.

http://www.liblfds.org

Blank Xavier
A: 

I found another solution written in c:

http://www.ddj.com/hpc-high-performance-computing/219500200

RED SOFT ADAIR
+1  A: 

What about boost.freelock?

teZeriusz