Use of agents to complete side-effects in STM transactions

Question

I'm aware that it is generally bad practice to put functions with side-effects within STM transactions, as they can potentially be retried and called multiple times.

It occurs to me however that you could use agents to ensure that the side effects get executed only after the transaction successfully completes.

e.g.

(dosync
  // transactional stuff
  (send some-agent #(function-with-side-effects params))
  // more transactional stuff
  )

Is this good practice?

What are the pros/cons/pitfalls?

Iapetus

The one of the core ideas of STM is failure atomicity. How would this help with that? – Hayrick

The point is for side effects that need to occur after a transaction succeeds but aren't themselves part of the transaction, e.g. sending a confirmation email. Clearly, you don't want to do this each time the transaction retries or else you could get a very angry / confused recipient! – Iapetus

Answer 1

Original:

Seems like that should work to me. Depending on what your side effects are, you might want to use send-off (for IO-bound ops) instead of send (for cpu-bound ops). The send/send-off will enqueue the task into one of the internal agent executor pools (there is a fixed size pool for cpu and unbounded size pool for io ops). Once the task is enqueued, the work is off the dosync's thread so you're disconnected at that point.

You'll need to capture any values you need from within the transaction into the sent function of course. And you need to deal with that send possibly occurring multiple times due to retries.

Update (see comments):

Agent sends within the ref's transaction are held until the ref transaction successfully completes and are executed once. So in my answer above, the send will NOT occur multiple times, however it won't occur during the ref transaction which may not be what you want (if you expect to log or do side-effecty stuff).

Osmose

@mikera, @Alex, sending is guaranteed to occur once only after the transaction succeeds, and shouldn't occur multiple times. (Statement 5 in clojure.org/agents) – Bedstraw

@Bedstraw - I think you're talking about something else. That statement refers to how the function passed in a send/send-off will be executed. What I am saying is that by making that call inside the ref transaction, the ref transaction may be retried causing multiple send/send-off calls, each of which executes exactly once. – Osmose

@alex, excuse me, I referenced the wrong point, further in the text, it says "Agents are integrated with the STM - any dispatches made in a transaction are held until it commits, and are discarded if it is retried or aborted." – Bedstraw

@Bedstraw hmmm....I see your point. I'm still not sure I agree that's what will happen as I don't know that a send in a ref's transaction will have that same constraint. Might be time to write a little code to find out... – Osmose

@Bedstraw indeed, you are correct. An agent send from within the ref transaction will be held and executed at the end of the successful ref transaction. So, @Iapetus my original answer above was incorrect. – Osmose

Thanks guys - extremely helpful and seems to do exactly what I want (i.e. side effect triggered only once if and only if the transaction succeeds) – Iapetus

Answer 2

This works and is common practice. However, like Alex rightly pointed out you should consider send-off over send.

There are more ways to capture commited-values and hand them out of the transaction. For example you can return them in a vector (or a map or whatever).

(let [[x y z] (dosync
                ; do stuff
                [@x @y @z])] ; values of interest to sode effects
  (side-effect x y z))

or you can call reset! on a local atom (defined outside the lexical scope of the dosync block of course).

Cottonmouth

Answer 3

There's nothing wrong with using agents, but simply returning from the transaction values needed for the side-effecting computation is often sufficient.

Refs are probably the cleanest way to do this, but you can even manage it with just atoms!

(def work-queue-size (atom [0]))

(defn add-job [thunk]
  (let [[running accepted?]
        (swap! work-queue-size
               (fn [[active]]
                 (if (< active 3)
                   [(inc active) true]
                   [active false])))]
    (println
     (str "Your job has been "
          (if accepted?
            "queued, and there are "
            "rejected - there are already ")
          running
          " total running jobs"))))

The swap! can retry as many times as needed, but the work queue will never get larger than three, and you will always print exactly once a message that is tied correctly to the acceptance of your work item. The "original design" called for just a single int in the atom, but you can turn it into a pair in order to pass interesting data back out of the computation.

Mawkin