Flatten Scala Try

Question

Is there a simple way to flatten a collection of try's to give either a success of the try values, or just the failure? For example:

def map(l:List[Int]) = l map {
  case 4 => Failure(new Exception("failed"))
  case i => Success(i)
}

val l1 = List(1,2,3,4,5,6)
val result1 = something(map(l1))

result1: Failure(Exception("failed"))

val l2 = List(1,2,3,5,6)
val result2 = something(map(l2)) 

result2: Try(List(1,2,3,5,6))

And can how would you handle multiple Failures in the collection?

Aetolia

Please say how your result should look like. List(1, 2, 3, Exception, 5, 7)? – Middelburg

Scalaz would call this operation sequence, but it's sadly missing from the standard library. – Anta

Answer 1

Maybe not as simple as you hoped for, but this works:

def flatten[T](xs: Seq[Try[T]]): Try[Seq[T]] = {
  val (ss: Seq[Success[T]]@unchecked, fs: Seq[Failure[T]]@unchecked) =
    xs.partition(_.isSuccess)

  if (fs.isEmpty) Success(ss map (_.get))
  else Failure[Seq[T]](fs(0).exception) // Only keep the first failure
}

val xs = List(1,2,3,4,5,6)
val ys = List(1,2,3,5,6)

println(flatten(map(xs))) // Failure(java.lang.Exception: failed)
println(flatten(map(ys))) // Success(List(1, 2, 3, 5, 6))

Note that the use of partition is not as type safe as it gets, as witnessed by the @unchecked annotations. In that respect, a foldLeft that accumulates two sequences Seq[Success[T]] and Seq[Failure[T]] would be better.

If you wanted to keep all failures, you can use this:

def flatten2[T](xs: Seq[Try[T]]): Either[Seq[T], Seq[Throwable]] = {
  val (ss: Seq[Success[T]]@unchecked, fs: Seq[Failure[T]]@unchecked) =
    xs.partition(_.isSuccess)

  if (fs.isEmpty) Left(ss map (_.get))
  else Right(fs map (_.exception))
}

val zs = List(1,4,2,3,4,5,6)

println(flatten2(map(xs))) // Right(List(java.lang.Exception: failed))
println(flatten2(map(ys))) // Left(List(1, 2, 3, 5, 6))
println(flatten2(map(zs))) // Right(List(java.lang.Exception: failed, 
                           //            java.lang.Exception: failed))

Middelburg

+1 " In that respect, a foldLeft that accumulates two sequences Seq[Success[T]] and Seq[Failure[T]] would be better. " Yes ! There is so many validation or failures types in the scala ecosystem (Option, Either, Try, scalaz.Disjunction, scalaz.Validation,...), and still I couldn't find any such double seq fold exemples among the public methods and common patterns. – Homager

Answer 2

This is pretty close to minimal for fail-first operation:

def something[A](xs: Seq[Try[A]]) =
  Try(xs.map(_.get))

(to the point where you shouldn't bother creating a method; just use Try). If you want all the failures, a method is reasonable; I'd use an Either:

def something[A](xs: Seq[Try[A]]) =
  Try(Right(xs.map(_.get))).
  getOrElse(Left(xs.collect{ case Failure(t) => t }))

Volcano

Nice! (Although it re-throws the exception, if any) How would you extend it such that all exceptions are kept? – Middelburg

@mhs - If failure is uncommon or performance is not critical, you want the re-throw as it's the simplest way to deal with the problem. The exception's stack trace has already been generated, so the throw/catch itself isn't too expensive. If I wanted to keep all exceptions I'd pack them in an Either instead, e.g. as in my edit. – Volcano

Answer 3

A little less verbose, and more type safe:

def sequence[T](xs : Seq[Try[T]]) : Try[Seq[T]] = (Try(Seq[T]()) /: xs) {
    (a, b) => a flatMap (c => b map (d => c :+ d))
}

Results:

sequence(l1)

res8: scala.util.Try[Seq[Int]] = Failure(java.lang.Exception: failed)

sequence(l2)

res9: scala.util.Try[Seq[Int]] = Success(List(1, 2, 3, 5, 6))

Anta

It looks nice, though is there not going to be quite a performance hit with the append operation? – Aetolia

Fair point - I'd forgotten that the default Seq is a List. Add reverse/use a vector at your discretion! – Anta

Nice, I hadnt seen that foldLeft shortcut before /: – Rabbinate

Answer 4

As an addition to Impredicative's answer and comment, if you have both scalaz-seven and scalaz-contrib/scala210 in your dependencies:

> scala210/console
[warn] Credentials file /home/folone/.ivy2/.credentials does not exist
[info] Starting scala interpreter...
[info] 
Welcome to Scala version 2.10.0 (OpenJDK 64-Bit Server VM, Java 1.7.0_17).
Type in expressions to have them evaluated.
Type :help for more information.

scala> import scala.util._
import scala.util._

scala> def map(l:List[Int]): List[Try[Int]] = l map {
     |   case 4 => Failure(new Exception("failed"))
     |   case i => Success(i)
     | }
map: (l: List[Int])List[scala.util.Try[Int]]

scala> import scalaz._, Scalaz._
import scalaz._
import Scalaz._

scala> import scalaz.contrib.std.utilTry._
import scalaz.contrib.std.utilTry._

scala> val l1 = List(1,2,3,4,5,6)
l1: List[Int] = List(1, 2, 3, 4, 5, 6)

scala> map(l1).sequence
res2: scala.util.Try[List[Int]] = Failure(java.lang.Exception: failed)

scala> val l2 = List(1,2,3,5,6)
l2: List[Int] = List(1, 2, 3, 5, 6)

scala> map(l2).sequence
res3: scala.util.Try[List[Int]] = Success(List(1, 2, 3, 5, 6))

You need scalaz to get an Applicative instance for the List (hidden in the MonadPlus instance), to get the sequence method. You need scalaz-contrib for the Traverse instance of Try, which is required by the sequence's type signature. Try lives outside of scalaz, since it only appeared in scala 2.10, and scalaz aims to cross-compile to earlier versions).

Raji

Answer 5

Starting in Scala 2.13, most collections are provided with a partitionMap method which partitions elements based on a function returning either Right or Left.

In our case we can call partitionMap with a function that transforms our Trys into Eithers (Try::toEither) in order to partition Successes as Rights and Failures as Lefts.

Then it's just a matter of matching the resulting partitioned tuple of lefts and rights based on whether or not there are lefts:

tries.partitionMap(_.toEither) match {
  case (Nil, rights)       => Success(rights)
  case (firstLeft :: _, _) => Failure(firstLeft)
}
// * val tries = List(Success(10), Success(20), Success(30))
//       => Try[List[Int]] = Success(List(10, 20, 30))
// * val tries = List(Success(10), Success(20), Failure(new Exception("error1")))
//       => Try[List[Int]] = Failure(java.lang.Exception: error1)

Details of the intermediate partitionMap step:

List(Success(10), Success(20), Failure(new Exception("error1"))).partitionMap(_.toEither)
// => (List[Throwable], List[Int]) = (List(java.lang.Exception: error1), List(10, 20))

Correctitude

Answer 6

Have a look on the liftweb Box monad. With the help of the tryo constructor function, it gives you exactly the abstraction you are looking for.

With tryo you can lift a function into a Box. The box then either contains the result from the function or it contains an error. You can then access the box with the usual monadic helper functions (flatMap, filter, etc.), without bothering if the box contains an error or the result form the function.

Example:

import net.liftweb.util.Helpers.tryo

List("1", "2", "not_a_number") map (x => tryo(x.toInt)) map (_ map (_ + 1 ))

Results to

List[net.liftweb.common.Box[Int]] = 
  List(
    Full(2), 
    Full(3), 
    Failure(For input string: "not_a_number",Full(java.lang.NumberFormatException: For input string: "not_a_number"),Empty)
  )

You can skip the erroneous values with flatMap

List("1", "2", "not_a_number") map (x => tryo(x.toInt)) flatMap (_ map (_ + 1 ))

Results

List[Int] = List(2, 3)

There are multiple other helper methods, e.g. for combining boxes (while chaining error messages). You can find a good overview here: Box Cheat Sheet for Lift

You can use it on its own, no need to use the whole lift framework. For the examples above I used the follwing sbt script:

scalaVersion := "2.9.1"

libraryDependencies += "net.liftweb" %% "lift-common" % "2.5-RC2"

libraryDependencies += "net.liftweb" %% "lift-util" % "2.5-RC2"

Rhizoid

Answer 7

These are my 2cents:

def sequence[A, M[_] <: TraversableOnce[_]](in: M[Try[A]])
  (implicit cbf:CanBuildFrom[M[Try[A]], A, M[A]]): Try[M[A]] = {
    in.foldLeft(Try(cbf(in))) {
      (txs, tx) =>
        for {
          xs <- txs
          x <- tx.asInstanceOf[Try[A]]
        } yield {
          xs += x
        }
    }.map(_.result())
  }

Barreto