When to use core.async in Clojure?

Question

When should I use Clojure's core.async library, what kind of applications need that kinda async thing?

Clojure provides 4 basic mutable models like refs, agents, atoms and thread locals/vars. Can't these mutable references provide in any way what core.async provides with ease?

Could you provide real world use cases for async programming?

How can I gain an understanding of it so that when I see a problem, it clicks and I say "This is the place I should apply core.async"?

Also we can use core.async in ClojureScript which is a single threaded environment, what are the advantages there (besides avoiding callback hell)?

Retention

Don't forget you can also use a future or promise. – Hampshire

Answer 1

You may wish to read this:

• Clojure core.async Channels Introductory blog by Rich Hickey
• Mastering Concurrent Processes with core.async Brave Clojure entry

The best use case for core.async is ClojureScript, since it allows you to simulate multi-threaded programming and avoid Callback Hell.

In JVM Clojure, core.async can also by handy where you want a (lightweight) producer-consumer architecure. Of course, you could always use native Java queues for that, as well.

Hampshire

A comment from someone that's working in a quite big project in ClojureScript: don't go there. The semantics are different, and it can break the JS runtime. Prefer promises, or other mechanisms - using instead of promises can "bubble" an exception to the top-level and break the runtime; using instead of callbacks can lead to "Maximum number of pending put!" error... – Labourer

Answer 2

It's important to point out that there are 2 common meanings associated to the word 'async' in programming circles:

1. asynchronous messaging: Systems in which components send messages without expecting a response from their consumers, and often without even knowing who the consumers are (via queues)
2. non-blocking (a.k.a event-driven) I/O: Programs structured in such a way that they don't block expensive computational resources (threads, cores, ...) while awaiting a response. There are several approches, with varying levels of abstraction, for dealing with such systems: callback-based APIs (low-level, difficult to manage because based on side-effects), Promise/Futures/Deferreds (representations of values that we don't yet have, more manageable as they are value-based), Green Threads ('logical' threads which emulate ordinary control flow, but are inexpensive)

core.async is very opinionated towards 1 (asynchronous messaging via queues), and provides a macro for implementing Green Threading (the go macro).

From my experience, if non-blocking is all you need, I would personally recommend starting with Manifold, which makes fewer assumptions about your use case, then use core.async for the more advanced use cases where it falls short; note that both libraries interoperate well.

Catron

Answer 3

I find it useful for fine-grained, configurable control of side-effect parallelism on the JVM.

e.g. If the following executes a read from Cassandra, returning an async/chan:

(arche.async/execute connection :key {:values {:id "1"}})

Then the following performs a series of executes in parallel, where that parallelism is configurable, and the results are returned in-order.

(async/pipeline-async
    n-parallelism
    out
    #(arche/execute connection :key {:values %1 :channel %2})
    (async/to-chan [{:id "1"} {:id "2"} {:id "3"} ... ]))

Probably quite particular to my niche, but you get the idea.

• https://github.com/troy-west/arche

Arielariela

Answer 4

Core.async provides building blocks for socket like programming which is useful for coordinating producer/consumer interaction in Clojure. Core.async's lightweight threads (go blocks) let you write imperative style code reading from channels instead of using callbacks in the browser. On the JVM, the lightweight threads let you utilize your full CPU threads.

You can see an example of a full-stack CLJ/CLJS producer/consumer Core.async chat room here: https://github.com/briangorman/hablamos

Another killer feature of core.async is the pipeline feature. Often in data processing you will have the initial processing stage take up most of the CPU time, while later stages e.g. reducing will take up significantly less. With the async pipeline feature, you can split up the processing over channels to add parallelization to your pipeline. Core.async channels work with transducers, so channels play nice with the rest of the language now.

Pomfrey