Is it possible to use Go's buffered channel as a thread-safe queue?

Question

I want to find a queue structure (a data container) whose elements must be first-in-first-out. It is important for me that the structure must be thread-safe. I'm going to use this data container as something like a task or connection pool.

I know a buffered channel is thread-safe, but I wonder if it works as FIFO, especially in a concurrent situation.

And if it is possible to use buffered channel as a thread-safe queue, do I need to worry about its efficiency?

Stopgap

Channels are the answer. There's little chance of finding or writing something better than channels for such a task. – Bergen

Channels are especially suited to tasks queues, resources sharing, connection pools, and the like. Be careful not to reproduce the patterns of a thread based language as one of the big progress of the Go language is the goroutine+channel mechanism. Be sure to understand it (you probably can submit an architecture or strategy to SO). – Playbill

This is the first thing I thought of when I learned about channels in Go, thanks for asking this! – Bookcraft

But the are a question, I think if use golang channels will be at most times, less secure, standard and complete than use a message broker like RabbitMQ or ActiveMQ with the same effort at the end. So I do not think that is a compensative aproach – Scotch

Answer 1

I'm pretty sure that Channels are FIFO. They are also cheap so they would be memory efficient. Beyond that without knowing the details of how you are going to use them We can't really give much more advice.

Robbins

Thanks. So, the Channels are both FIFO and Thread-Safe? Would you please give me a link or something about the implementation details of Channel's mechanism? – Stopgap

The definitive documentation would be: code.google.com/p/go/source/browse/src/pkg/runtime/chan.c See the implementation of chansend and chanrecv for the details. The specification does not specifically say they are a fifo queue but it's derivable I think from the details of how to use them. – Robbins

But the are a question, I think if use golang channels will be at most times, less secure, standard and complete than use a message broker like RabbitMQ or ActiveMQ with the same effort at the end. So I do not think that is a compensative aproach – Scotch

Answer 2

In Go, a buffered channel is just that: a thread-safe FIFO queue so what you are trying to do is perfectly valid. You shouldn't have performance issues at all with this approach.

Sommelier

The Go Programming Language asks to avoid use Buffered Channels as queues and use slices instead. – Ba

@Ba could you link to this by chance? I've never heard of this. It's hard to avoid using Buffered Channels as queues, because that's effectively what they are... I'm wondering if there's more nuance here. – Incentive

@Incentive "Novices are sometimes tempted to use buffered channels within a single goroutine as a queue, lured by their pleasingly simple syntax, but this is a mistake . Channels are deeply connected to goroutine scheduling, and without another goroutine receiving from the channel,a sender—and perhaps the whole program—risks becoming blocked forever. If all you need is a simple queue, make one using a slice." P233 – Rhpositive

The case of the book refer to is where only one goroutine in the whole program. – Rhpositive

@Rhpositive Thinking of how queues generally work in other languages, such as C++, the same is still true. If you're implementing a queue to feed a pool of threads, if the sender stops sending the receivers are usually block indefinitely on a cond_var. You can notify_all the cond_var to get the threads unstuck to do some kind of cleanup logic, which is synonymous to having a context or a control channel in golang with a select statement. So really depends on what you're using the queue for, if it's a work stealing queue, use a channel. – Incentive

I suppose many of us are probably assuming the need for this queue is likely work stealing which could be wrong, but that said it's really situational weather or not you'd be concerned with the way a buffered channel works. Empty channel blocks receivers, full channel blocks senders. If that behavior is undesirable, probably don't use a channel. – Incentive

Answer 3

In general, I would say buffered channels do not make a good concurrency-safe queue. Creating them allocates memory for the entire buffer. If your queue size varies from very small to very large during execution, you have to allocate for the worst case scenario and may be wasting a lot of memory.

Stilbestrol

This is a very broad assumption. On the other side if you have a dynamically sized piece of memory you're constantly having to reallocate. If you want a buffered queue, you're probably acutely aware of the buffer size and tolerate the space. Similarly if you're going to support some worst case scenario where you might expect some buffer to become big enough to fill that space then it should be safe to assume you can allocate all of that memory up front. If you can't allocate that memory up front, then you probably can't allocate that much memory at all, and need a different solution. – Incentive