Akka: correct pattern for syncing state between two actors

Question

In the process of rewriting the deprecated persistentView in Akka, I have a read actor that needs to snapshot its state and the journal events offset is has read so far. To stop the view actor requiring serialisation of the composite data structure of: (state + offset), I am delegating the responsibility of snapshotting the offset value to a child actor. The problem then is syncing the offset value between these two. Currently in the read actor, i have:

case RecoveryCompleted ⇒
  implicit val ec = context.dispatcher
  val lastSequenceNr = (sequenceSnapshotterRef ? GetLastSnapshottedSequenceNr).mapTo[QueryOffset]
  lastSequenceNr onComplete {
    case Success(QueryOffset(sequenceNr)) ⇒
      offsetForNextFetch = sequenceNr
      doSomethingBasedOnThecompositeData()
      ...

and to update the child actor's offset value in order to sync the snapshots, I do:

case RequestSnapshot ⇒
  implicit val ec = context.dispatcher
  val offsetUpdated = sequenceSnapshotterRef ? 
QueryViewSequenceApi.UpdateSequenceNr(offsetForNextFetch)

  offsetUpdated map {
    _ ⇒
      saveSnapshot()
      snapshotRequested = false
  } recover{
    case _ ⇒
      self ! RequestSnapshot
      log.debug("QueryViewSequenceSnapshotter not reachable. Will try again.")
  }
}

This however means that if the child actor's acknowledgement were to be lost or if the child actor died before sending the message, and then the view actor dies while waiting for the offsetUpdated response from the child actor, the offset and the state will be in an unsynced state when the parent actor tries to recover.

• Is this case even worth worrying about? would a local child actor randomly die if it is simply doing simple arithmetic as is the case for my child actor?
• How can I modify the design to ensure that can be handled? I could potentially introduce a acknowledgement on both sides and introduce a two-staged syncing mechanism on both sides, but that's likely to buggy.

Here is the full context of the question.

Update: Reading some more on message delivery reliability, I realise that this is a more general problem. Can I configure this parent and actor to use a at-least-once delivery mechanism while the rest of my project uses the default at-most-once delivery mechanism? that still won't address the problem of the child actor dying before attempting to send the acknowledgement message back to the parent.

Asquith

Answer 1

I would suggest using ask for the persistence guarantee, as well as Akka's death watch. Its still not trivial : Lots of loopholes present still.

Akka also has exactly-once guarantee, but in general, what I do : Fire away the request until confirmation is done. Regular tell, and I keep state in the sender. Timeout is handled by a scheduler.

If the child dies, react to the Terminated, and just keep the loop going.

Algor