Already existing functional way for a retry-until in Scala?

Question

Is there a functional/Scala way to call a function repeatedly until it succeeds, while reacting to failed attempts?

Let me illustrate with an example. Suppose I want to read an integer from standard-in, and retry if the user did not in fact enter an integer.

Given this function:

def read_int(): Either[String, Int] = {
  val str = scala.io.StdIn.readLine()
  try {
    Right(str.trim().toInt)
  } catch {
    case _: java.lang.NumberFormatException => Left(str)
  }
}

And this anonymous functions:

val ask_for_int = () => {
  println("Please enter an Int:")
  read_int()
}

val handle_not_int = (s: String) => {
  println("That was not an Int! You typed: " + s)
}

I would use them like this:

val num = retry_until_right(ask_for_int)(handle_not_int)
println(s"Thanks! You entered: $num")

My questions are:

• Does something like retry_until_right already exist in Scala?
• Could it be solved with existing facilities? (Streams, Iterators, Monads, etc.)
• Do any of the FP libraries (scalaz?) offer something like this?
• Could I do anything better / more idiomatic? (*)

Thanks!

*) Apart from the snake_case. I really like it.

Quenna

possible duplicate of What's the Scala way to implement a retry-able call like this one? – Feckless

I don't think my question is a duplicate of that one. That one is specific to failure by exception, and the amount of retries are numbered. My question is more abstract. The other question could be a particular case of mine. That question is also "what is the best way to implement it?", mine is "do I have to implement it?" (or does it exist already?) – Quenna

Answer 1

This was my first tail-recursive implementation:

@scala.annotation.tailrec
def retry_until_right[WRONG, RIGHT](generator: () => Either[WRONG, RIGHT])(on_wrong: WRONG => Any): RIGHT = {
  generator() match {
    case Right(right) =>
      right
    case Left(wrong) =>
      on_wrong(wrong)
      retry_until_right(generator)(on_wrong)
  }
}

But wanting to re-use existing libraries, I then switched to this approach using iterators:

def retry_until_right[WRONG, RIGHT](generator: => Either[WRONG, RIGHT])(on_wrong: WRONG => Any): RIGHT =
  Iterator.continually(generator).flatMap {
    case Left(value) =>
      on_wrong(value)
      None
    case Right(value) =>
      Some(value)
  }.toSeq.head

Which can be used like this:

val num = retry_until_right(ask_for_int()) { str =>
  println("Ivalid input: " + str)
}
println("Thanks! You entered: " + num)

However, it could be argued that hiding the Iterator from plain view inside the implementation can be inflexible. What if the developer wants to perform further operations on it? (mapping, etc.)

Instead, the operations that react on "wrong" values and finally select the first "right" one can be abstracted into an "extension" class specific for iterators of the Either[L,R] type, like this:

implicit class EitherIteratorExtensions[L, R](it: Iterator[Either[L, R]]) {
  def onLeft(callback: L => Any) =
    it.map {
      case left @ Left(value) =>
        callback(value)
        left
      case right => right
    }

  // take only Right elements
  def takeRight: Iterator[R] = 
    it.flatMap {
      case Left(_) =>
        None
      case Right(value) => Some(value)
    }

  // iterate and fetch the first Right element
  def firstRight: R = {
    takeRight.toSeq.head
  }
}

Now we can comfortably use the needed methods, in succinct code, while retaining control of the Iterator like this:

val num = Iterator.continually(ask_for_int()).onLeft(handle_not_int).firstRight

println("Thanks! You entered: " + num)

Though I'm happy with this approach, I still wonder if this is not part of an existing library already...

Quenna

Won't the toSeq produce an infinite collection? Perhaps you want a BufferedIterator so you can have the head? – Eyrir

@Eyrir - I think it does not. toList seems to force the collection, not toSeq. But I might be mistaken. At least I tried it, and it works. – Quenna

Also note that it can not succeed in producing an infinite collection, because it depends on the input of ask_for_int(). It has to wait for it. It would result in an infinite loop if ask_for_int() never returned an Int. But that's the idea anyway. – Quenna

Answer 2

I think the Try monad together with the Iterator.continually method is suitable for this general problem. Of course, this answer could be adopted to use Either if you are so inclined:

def retry[T](op: => Try[T])(onWrong: Throwable => Any) = 
    Iterator.continually(op).flatMap { 
        case Success(t) => Some(t)
        case Failure(f) => onWrong(f); None 
    }.toSeq.head

Then you can do:

retry { Try(scala.io.StdIn.readLine.toInt) }{ _ => println("failed!") }

Or you can even hide the Try part of the implementation, and give onWrong a default value, and make it a second parameter instead of a curried function:

def retry[T](op: => T, onWrong: Throwable => Any = _ => ()) = 
    Iterator.continually(Try(op)).flatMap { 
        case Success(t) => Some(t)
        case Failure(f) => onWrong(f); None 
    }.toSeq.head

So then you can simply:

retry { scala.io.StdIn.readLine.toInt } { _ => println("failed") }

Or

retry { scala.io.StdIn.readLine.toInt }

Stereotomy

I see your approach is similar to mine (continually / flatMap / toSeq / head). Glad to see I was on the right track. So it seems this is not built-in. Maybe scalaz, shapeless or some of those have something like this... – Quenna

Answer 3

Here's an alternative solution using scalaz.concurrent.Task:

import scalaz.concurrent.Task

def readInt: Task[Int] = {
  Task.delay(scala.io.StdIn.readLine().trim().toInt).handleWith {
    case e: java.lang.NumberFormatException =>
      Task.delay(println("Failure!")) flatMap (_ => readInt)
  }
}

And a retry wrapper (a bit less flexible):

def retry[A](f: Task[A])(onError: PartialFunction[Throwable, Task[_]]): Task[A] =
  f handleWith (onError andThen (_.flatMap(_ => retry(f)(onError))))

val rawReadInt: Task[Int] = Task.delay(scala.io.StdIn.readLine().trim().toInt)

val readInt: Task[Int] = retry(rawReadInt) {
  case e: java.lang.NumberFormatException => Task.delay(println("Failure!"))
}

Explanation

scalaz.concurrent.Task[A] is a monadic structure that eventually returns a A. It uses a trampoline to (usually) avoid stack overflows. It also handles exceptions, and can either rethrow the exception, or represent the exception via \/ (scalaz's right-biased Either).

handleWith allows one to write a handler for a Throwable which was raised by the Task. The result of this handler is a new Task to run afterwards. In this case, we will just print out an error message, and call the original Task again using flatMap. Since Task is a trampolined construct, this should be safe.

Try this with readInt.run - this will run the task on the current thread, and eventually return the Int value passed in.

Schnabel

Sorry... newbie here... what is a "trampoline" in this context? – Quenna

Trampolining is a strategy for removing stack overflows; it represents the hierarchical computation as a data structure instead of a set of stack frames. The "trampoline" then is a execution framework which can process this data structure step-by-step, until the final result is evaluated. From the scaladoc for scalaz.concurrent.Future: "Future is a trampolined computation producing an A that may include asynchronous steps. Like Trampoline, arbitrary monadic expressions involving map and flatMap are guaranteed to use constant stack space." See also: "Functional Programming in Scala", Ch. 13. – Schnabel

Perhaps a better reference is the paper, "Stackless Scala With Free Monads", by Rúnar Óli Bjarnason: blog.higher-order.com/assets/trampolines.pdf – Schnabel

Answer 4

def retry[L, R](f: => Either[L, R])(handler: L => Any): R = {
    val e = f
    e.fold(l => { handler(l); retry(f)(handler) }, identity)
}

Schismatic

Very interesting use of folding and in particular identity. Was new to me. – Quenna

The identity function is a very natural thing to use when folding with functions @SebastianN. . In the same way that 0 is the natural seed when folding a sum and 1 when folding a product, identity is the natural seed where a fold would build a cumulative function. If the collection is empty, you receive a function that just returns its argument. If not, you receive an iterative function which transforms the argument (with identity being a harmless initial or final stage). Useful with collections, Option, Either and others. – Lasonyalasorella

Answer 5

An implementation of a retry monad can be found here: https://github.com/hipjim/scala-retry It has various retry strategies.

// define the retry strategy
implicit val retryStrategy =
    RetryStrategy.fixedBackOff(retryDuration = 1.seconds, maxAttempts = 2)

// pattern match the result
val r = Retry(1 / 1) match {
    case Success(x) => x
    case Failure(t) => log("I got 99 problems but you won't be one", t)
}

Pneumogastric