Clojure STM ( dosync ) x Java synchronize block

Question

What is the difference between Clojure STM (dosync) approach and Java synchronize Block?

I'm reading the code below from "The sleeping barber" problem. (http://www.bestinclass.dk/index.clj/2009/09/scala-vs-clojure-round-2-concurrency.html)

(defn the-shop [a]  
  (print "[k] entering shop" a)  
  (dosync     
    (if (< (count @queue) seats)  
      (alter queue conj a)  
      (print "[s] turning away customer" a))))

To avoid race conditions, dosync is used, so i ask myself "What is the difference (STM) from Java synchronize block" ? Will it block this critical code ?

Thanks in advance ! Dantas

Liggett

Hi, Michl, like u answered me about STM perfectly, let me ask another stuff about clojure that i dont realize. The Commute form. Suppose that i have 2 threads (t1 and t2 ). And the ideia is increment a reference, ok ? t1 inside a dosync, get the ref value - in transaction value ( 0 ) - ... t2 get control and was able to increment the reference from 0 to 1. t1 come back , and what its supposed to do ? wether t1 continues executing the final result will no be correct. how commute works in this case ? thanks in advance – Liggett

Answer 1

dosync and synchronized give access to completely different concurrency abstractions.

synchronized is a way of acquiring and releasing locks. When a thread enters a synchronized block, it attempts to acquire the appropriate lock; if the lock is currently held by a different thread, the current thread blocks and waits for it to be released. This leads to certain problems, such as the risk of deadlock. The lock is released when the thread leaves the synchronized block.

dosync marks a block of code which is to be run in a transaction. Transactions in Clojure are a way of coordinating changes to Refs (objects created with the ref function); if you need some code to have a consistent view of some pieces of mutable state in Clojure -- and possibly change them -- you put those in Refs and execute your code in a transaction.

A transaction has the interesting property that it will restart if for some reason it cannot commit, up to a certain maximal number of retries (currently hard-coded to be 10000). Among the possible reasons for a transaction being unable to commit are an inability to obtain a consistent view of the world (actually, the relevant Refs -- there is an "adaptive history" facility which makes this less of a problem than it might seem at first glance); simultaneous changes made by other transactions; etc.

A transaction runs no risk of being deadlocked (unless the programmer goes out of their way to introduce a deadlock unrelated to the STM system through Java interop); livelock, on the other hand, is a certain possibility, though it is not very probable. In general, many -- although not all! -- of the intuitions programmers associate with database transactions are valid in the context of STM systems, including that of Clojure.

STM is a huge topic; one excellent resource for learning about Clojure's STM is Mark Volkmann's Software Transactional Memory article. It goes into great depth in discussing Clojure's STM in its final sections, but the beginning can serve as great introductory reading.

As for the snippet you quoted, it's actually not something you would normally want to emulate in production code, since dosync blocks should almost always be side-effect free; the print here can be useful for demonstrating the inner working of the STM, but if you wanted a transaction to cause side-effects in real code, you should have it spawn a Clojure Agent for the purpose (which would only execute its task if the transaction successfully commits).

Superintend

Michal , u wrote "since dosync blocks should almost always be side-effect free" . a console print will break this "side-effect" property ? so update a database table will do the same, all right ? What is supposed be inside a dosync block ? Thanks again, excellent post ! – Liggett

Right, printing out to the console and updating a database table both qualify as side effects. dosync blocks are in general not supposed to contain side-effecty code except calls to transaction-aware Ref-modifying functions (alter / commute / ref-set), with the important exception that any actions sent to Agents may contain side effects meant to happen after the transaction completes (all such actions will be performed if and when the transaction commits, so there is no risk of causing the same side effect more then once). – Lornalorne

So, if you needed to modify the values of a couple of Refs and update your database, your dosync would contain alter / commute calls to change the Refs and a send / send-off telling an Agent to perform the db update once the transaction commits (and so no longer runs the risk of retrying or failing due to the retry limit being reached). – Lornalorne

Oh, also, about the "almost" -- debug printouts in transactions can be as useful as anywhere else (in particular, they will tell you how many times your transaction needed to retry, as Lau's "sleeping barber" code nicely demonstrates); calls to memoised functions from within transactions should of course cause the side effect of recording any newly computed values; etc. Still, one should be really sure of what one's doing when introducing side effects inside dosync -- lots of breakage can result from a lack of care on this point. – Lornalorne

Thanks Michał. Excellent view ! Really nice ! – Liggett

When you send-off an agent from within a dosync, it doesn't run until the dosync ends. This means it's too late to rollback the STM transaction if something goes wrong in your agent, which is always possible if your agent is accessing databases (the db might reject your data or the server might be down). I think coordinating a ref and a database is currently very hard (or impossible?) even using agents. – Barram

Oh, that is absolutely true...! Thanks for the comment. This just goes to confirm my suspicion that my STM-fu is a bit rusty... – Lornalorne

Answer 2

Also in addition to Michał's excellent answer, with STM transactions, reads always get you the frozen value at the beginning of the transaction and need not wait for any ongoing transaction to complete.

Scansion

Yes ! Read operations do not requires lock. it is lock free. when u read, u get a snapshot. But if u write, the operation only commit if the snapshot is equal from what u have. if when u try write the data isn´t the same, the operation rollback. – Liggett

Answer 3

Just to give a full picture for those seeking, Clojure does have a synchronized analog. It is useful when one has to work with Java non-threadsafe types for interop.

(locking x & body)

Autoroute

Answer 4

basic difference is following

Clojure STM supports optimistic concurrency whereas JAVA synchronized is pessimist

Clojure STM does not acquire lock until there are more than one thread. if mutable state is updated by another thread then the operation inside dosync is repeated. Also, dosync is mandatory for mutable states. Clojure throws illegalState exception when dosync is missing, unlike JAVA.

Floriaflorian

Well, they are quite different types of concurrency support (transactional access to mutable values vs. implementation of critical sections using synchronized) so they can't really be compared. If Java had STM implemented with two-phase locking then one could compare and say "that's an 'optimistic' protocol" (not 'optimistic concurrency', which sounds like a party) and "that's a 'pessimistic' protocol". It's a DBMS thing! And it's even down not to STM, but Clojure's MVCC implementation of it. – Calcify