Can this be done with STM?

Question

Disclaimer: This can easily be done using an MVar () as a simple mutex. I'm just curios to see whether it can be done with STM.

I want to do the following atomically:

• Read some variables.

• Decide what I/O to perform, based on what I just read.

• Perform the I/O.

• Record the results in the variables.

For concreteness, suppose I want to keep track of how many bytes of input I've read, and pretend I've reached EOF after a certain number of bytes have been consumed. (OK, letting two threads read from the same file concurrently is probably a bogus thing to do in the first place, but go with me on this...)

Clearly this cannot be a single STM transaction; there's I/O in the middle. Clearly it would also be wrong to have it as two unconnected transactions. (Two threads could see that there's one byte of quota left, and both decide to read that byte.)

Is there a nice solution to this problem? Or is STM simply the wrong tool for this task?

Auditory

Does this give you what you'd need? The "barber" could be your read pointer. #16934178 – Gae

@DaxFohl Well, you could use STM to send all the I/O requests to a single thread via a TChan... that ought to work. – Auditory

Seems like you've got a standard use case for the basic Erlang-style actor model solution, which is what the barber thing implements via STM and TChan. – Gae

Answer 1

I'd say STM can't do it, and it's on purpose. A piece of STM code can be restarted multiple times at various places if a transaction rolls back. What would happen if you run your transaction, it performs the I/O action and then rolls back when recording the results in the variables?

For this reason STM computations must be pure, only with the addition of STM primitives such as STM mutable variables and arrays.

Hudson

Certainly it would not make sense to perform I/O inside a transaction; you cannot undo I/O. But can transactions be used to achieve mutual exclusion? And is it sensible to do so? – Auditory

@Auditory As far as I know, mutual exclusion can't be done in STM. There are no guarantees that a STM framework will wait for anything - it can as well repeat a failed transaction over and over again in a loop until it succeeds. From the docs: The implementation may block the thread until one of the TVars that it has read from has been updated. Probably the best solution is in the answer given by comonad - as I understand it, it executes IO actions on a successful commit. – Hudson

@Auditory Just a thought, is it even possible to achieve both mutual exclusivity and composability without risking a deadlock? – Hudson

It is if there is only 1 resource being locked. But in the general case... no, this is unlikely to work. – Auditory

@Auditory Hmm, actually seems like there may be a monadic solution there--create a monad that, before executing, tries to grab all the mutexes of the functions that compose it from within an STM block. That way you get fine-as-you-want granularity but prevent deadlock, all for free just by putting your code in a do block. Seems reasonable, but certainly there's a flaw in my logic or someone would have done this already. – Gae

@DaxFohl I'd say you won't be able to enumerate the mutexes until you actually run the monad, because they could depend on values (results) during the computation. But it seems it'd be feasible with an Applicative where the effects are fixed and don't depend on values. – Hudson

Answer 2

Use a TVar Bool named consistent to keep track of whether your IO action is in progress. Before you run the IO action you set consistent to False and after you run the IO action you set consistent to True. Then, any action that depends on the values of those STM variables that you are modifying just puts this clause at the beginning:

do b <- readTVar consistent
   check b
   ...

This ensures that those actions only see a consistent view of the variables being modified and will not run while the IO action is in progress.

Etui

This has a flavor. Currently the problem I'm hitting is that a mutex action cannot run other mutex actions, because the mutex is already taken. I'm not sure if the above approach will solve this problem... – Auditory

Can you clarify the mutex problem? – Etui

If you use MVar () as a mutex, and begin every function with takeMVar, then one function cannot call another because the mutex is already taken. Functions become non-composable. – Auditory

@Auditory you could use TVar Option Integer, and only proceed if it's None or Some [currentThreadId] – Gae

@GabrielGonzalez I'm trying to write a module such that clients don't have to care about thread-safety. Each exported function takes the mutex, does stuff, releases it. But sometimes one exported function would like to call another exported function to do part of its work... which results in instant deadlock. It's not hard to work around, just mildly irritating. – Auditory

@Auditory actually you'd have to use something like TVar (Integer, Count); with TVar Option Integer your innermost function would release your before your outermost is finished. But all that said, I thought this is basically what MVar did. – Gae

@Auditory This is the general problem with lock-based programming, which is basically what this amounts to. I think you were probably more on the right track when you talked about sending all IO actions to some sort of channel in the actor style. It sounds like that might solve your problem better. – Etui

Answer 3

I think you are looking for the stm-io-hooks package.

Whatever you actually want to do – would it be possible to express that in terms of STM's abort/retry semantics? In other words: Can you do a rollback and repeat the IO action? If not, then I'd refer to the answer of Gabriel Gonzalez.

Longwinded

Well, all the I/O operations are read operations, so the only thing they modify is the state of a handle. If you could guarantee that no two I/O operations happen at the same time, and every one always seeks to the right place first... it might maybe work... But I think keeping I/O out of transactions would work better. – Auditory

@MathematicalOrchid: In that case I'd simply use different Handles. If that isn't an option, you would have to use a locking mechanism (MVar or Gabriel's answer) for seek+read. – Longwinded

@MathematicalOrchid: I'm not sure what you need STM for, but maybe you are only experimenting with concurrency. You do not need Mutexes like in lower languages: In Haskell you apply the locking mechanism directly to the resource, like MVar Handle instead of (MVar (),Handle). STM is a completely different way of solving concurrency issues, it uses "Transactions" that can be aborted/restored/repeated like a money transaction between bank accounts. I see no use in that for reading immutable files. – Longwinded

I get that. I'm using a mutex not to protect the handle, but to protect the other stuff I'm updating each time the handle is used. But, as I say, it ends up being rather non-composable. I can fix it, but I was just wondering whether STM offers a better way. I suspected not, but I wanted to check. – Auditory