STM and unsafePerformIO in Haskell

Question

The documentation for STM states that:

Using unsafePerformIO inside of atomically is also dangerous but for different reasons. See unsafeIOToSTM for more on this.

When it comes to using threads and async exceptions, there are functions to mask asynchronous exceptions so that resources can be safely allocated and freed.

But there are alot of functions that use unsafePerformIO behind the scenes, for example allocAndFreeze in the memory package, and it's not hard to force a thunk containing such an expression inside an STM transaction. Are those functions actually safe to use inside an STM transaction? Are there circumstances where it could lead to memory leaks or data corruption? Is there an equivalent of mask for this circumstance?

Thanks

Omnifarious

Answer 1

Ordinarily safe uses of unsafePerformIO may lead to resource leaks (not necessarily memory specifically) or data corruption if the IO is interrupted by an STM retry. This is for two reasons: first, STM retries do not run exception handlers, so if the unsafe IO relies on exception handlers to release resources (e.g. with bracket), they will not be cleaned up; and second, the IO may be interrupted at any point or executed multiple times, so it’s up to you to ensure it maintains program invariants even if interrupted.

So for example allocAndFreeze will not leak, because it uses ForeignPtr internally, which in GHC is just pinned memory in the managed heap, so it doesn’t rely on exception handlers or finalizers to reclaim the memory. However, it may cause data corruption in the sense that, if the unsafe IO temporarily breaks invariants in a data structure such as “the allocated array must always be sorted”, that breakage may become visible if the computation is interrupted at that point.

Lowtension

Thanks John! Can you confirm that an STM retry can interrupt an FFI call too? So, it could interrupt a call to a C routine in the middle of writing some data? – Omnifarious

@Scott: I’m not intimately familiar with the implementation of STM, so I’m not certain. I expect that if a retry happens during an uninterruptible foreign call, it will still wait for the call to return, but for an interruptible foreign call, I don’t know what will happen, and it might be undefined. – Lowtension

@Scott: A safer way to do a foreign call inside STM might be to not, and do any foreign calls outside STM, e.g. atomically setup; foreignCall; atomically teardown where setup and teardown take and release a lock of some kind (such as a TVar Bool); or you could execute it in a separate thread, so even if the transaction is retried, the call will complete normally, something like uninterruptibleUnsafeIOToSTM io = unsafeIOToSTM (do { m <- newEmptyMVar; forkIO ((putMVar m . Right =<< io) `catch` \ (e :: SomeException) -> putMVar m (Left e)); either throw pure =<< takeMVar m }) – Lowtension

@john, absolutely, but I thought it would be a good question to ask since information on the topic is quite sparse. The details of how Haskell aborts a computation in the case of an STM retry would be a welcome addition to the documentation IMO, and I think that as a library author it would be a good thing to be aware of. It's not obvious that, if you use unsafePerformIO, even if you cover async exceptions using something like bracket, your computation might still get interrupted if it's happening in STM. The uninterruptableUnsafeIOToSTM would be a good workaround in some cases. Thank you! – Omnifarious