How to convert Map[A,Future[B]] to Future[Map[A,B]]?

Question

I've been working with the Scala Akka library and have come across a bit of a problem. As the title says, I need to convert Map[A, Future[B]] to Future[Map[A,B]]. I know that one can use Future.sequence for Iterables like Lists, but that doesn't work in this case.

I was wondering: is there a clean way in Scala to make this conversion?

Fixer

I'd imagine Scalaz would have a Traversable (the Haskell sense of the word) instance for Map, which would do exactly what you need. – Cognomen

@Cognomen Now I will check SO to see if anyone has posted, What is the Haskell sense of the word Traversable? – Overhear

See these papers for the original formulation. The Scalaz version is named Traverse—see my answer below for how you'd use it here. – Bawdy

Answer 1

See if this works for you:

val map = Map("a" -> future{1}, "b" -> future{2}, "c" -> future{3})    
val fut = Future.sequence(map.map(entry => entry._2.map(i => (entry._1, i)))).map(_.toMap)

The idea is to map the map to an Iterable for a Tuple of the key of the map and the result of the future tied to that key. From there you can sequence that Iterable and then once you have the aggregate Future, map it and convert that Iterable of Tuples to a map via toMap.

Now, an alternative to this approach is to try and do something similar to what the sequence function is doing, with a couple of tweaks. You could write a sequenceMap function like so:

def sequenceMap[A, B](in: Map[B, Future[A]])(implicit executor: ExecutionContext): Future[Map[B, A]] = {
  val mb = new MapBuilder[B,A, Map[B,A]](Map())
  in.foldLeft(Promise.successful(mb).future) {
    (fr, fa) => for (r <- fr; a <- fa._2.asInstanceOf[Future[A]]) yield (r += ((fa._1, a)))
  } map (_.result)
}

And then use it in an example like this:

val map = Map("a" -> future{1}, "b" -> future{2}, "c" -> future{3})    
val fut = sequenceMap(map)
fut onComplete{
  case Success(m) => println(m)
  case Failure(ex) => ex.printStackTrace()
}

This might be slightly more efficient than the first example as it creates less intermediate collections and has less hits to the ExecutionContext.

Hurwitz

In the same spirit: Future.sequence(map map {case (a, b) => b map (a -> _ )}) map (_.toMap) – Actress

@MarkusMikkolainen, you could certainly make it more readable by expanding the code to multiple lines instead of the one liner I submitted. As far as efficiency, you will incur n+1 additional hits (additional as in above and beyond what happens with sequence) to the ExecutionContext (the +1 comes from the _.toMap at the end). Shouldn't be too big a deal, but I will try and post a solution that more mirrors the sequence function w/o using it directly for better efficiency in terms of hitting the ExecutionContext. – Hurwitz

@MarkusMikkolainen, okay, updated with another solution, one that mirrors the existing sequence functionality. – Hurwitz

@ChrisGrimm, to the best of my knowledge the combinators on Future like map, flatMap, recover etc will not cause blocking behavior. I believe that each just creates a new Promise that is completed in the onComplete for the preceeding Future and the Future from that Promise is what's returned as the result of the combinator. So in essence you are just chaining onCompletes together. – Hurwitz

@cmbaxter: Thanks a lot for this answer. It has really helped me to understand the nature of map/flatMap/futures in scala. :) – Fixer

+1 I think we should use more builders this way, balanced by DRY etc. I read the first shot yesterday and agree w/comment on readable efficiency. – Overhear

Why do you need to write fa._2.asInstanceOf[Future[A]]? Why isn’t fa._2 type already inferred to Future[A]? – Belden

@JulienRichard-Foy, good question. I actually took that code from the source of the scala library Future.sequence method and re-purposed it. That cast is in that code and I left it in there even though it seems it's not necessary. – Hurwitz

See this discussion for an explanation of this cast: groups.google.com/forum/#!topic/scala-user/YpLe7gOvrIo – Belden

Answer 2

I think the most succinct we can be with core Scala 2.12.x is

val futureMap = Map("a" -> future{1}, "b" -> future{2}, "c" -> future{3}) 

Future.traverse(futureMap.toList) { case (k, fv) => fv.map(k -> _) } map(_.toMap)

Ingham

Did traverse change overtime? I cannot get this code to work. I'm on scala 2.12 – Paphian

@Paphian I updated it. I don't write Scala much anymore so I can't say exactly why it broke. Including this in case it's useful: scalafiddle.io/sf/LBTLWYn/0. – Ingham

Answer 3

Update: You can actually get the nice .sequence syntax in Scalaz 7 without too much fuss:

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.{ Future, future }

import scalaz._, Scalaz.{ ToTraverseOps => _, _ }
import scalaz.contrib.std._

val m = Map("a" -> future(1), "b" -> future(2), "c" -> future(3))

And then:

scala> m.sequence.onSuccess { case result => println(result) }
Map(a -> 1, b -> 2, c -> 3)

In principle it shouldn't be necessary to hide ToTraverseOps like this, but for now it does the trick. See the rest of my answer below for more details about the Traverse type class, dependencies, etc.

---

As copumpkin notes in a comment above, Scalaz contains a Traverse type class with an instance for Map[A, _] that is one of the puzzle pieces here. The other piece is the Applicative instance for Future, which isn't in Scalaz 7 (which is still cross-built against pre-Future 2.9), but is in scalaz-contrib.

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import scalaz._, Scalaz._
import scalaz.contrib.std._

def sequence[A, B](m: Map[A, Future[B]]): Future[Map[A, B]] = {
   type M[X] = Map[A, X]
   (m: M[Future[B]]).sequence
}

Or:

def sequence[A, B](m: Map[A, Future[B]]): Future[Map[A, B]] =
  Traverse[({ type L[X] = Map[A, X] })#L] sequence m

Or:

def sequence[A, B](m: Map[A, Future[B]]): Future[Map[A, B]] =
  TraverseOpsUnapply(m).sequence

In a perfect world you'd be able to write m.sequence, but the TraverseOps machinery that should make this syntax possible isn't currently able to tell how to go from a particular Map instance to the appropriate Traverse instance.

Bawdy

Btw, SIP-14 is included in Scala 2.9.3 – Teniacide

Answer 4

This also works, where the idea is to use the sequence result (of the map's values) to fire a promise that says you can start retrieving values from your map. mapValues gives you a non-strict view of your map, so the value.get.get is only applied when you retrieve the value. That's right, you get to keep your map! Free ad for the puzzlers in that link.

import concurrent._
import concurrent.duration._
import scala.util._
import ExecutionContext.Implicits.global

object Test extends App {
  def calc(i: Int) = { Thread sleep i * 1000L ; i }
  val m = Map("a" -> future{calc(1)}, "b" -> future{calc(2)}, "c" -> future{calc(3)})
  val m2 = m mapValues (_.value.get.get)
  val k = Future sequence m.values
  val p = Promise[Map[String,Int]]
  k onFailure { case t: Throwable => p failure t }
  k onSuccess { case _ => p success m2 }
  val res = Await.result(p.future, Duration.Inf) 
  Console println res
}

Here's the REPL where you see it force the m2 map by printing all its values:

scala> val m2 = m mapValues (_.value.get.get)
m2: scala.collection.immutable.Map[String,Int] = Map(a -> 1, b -> 2, c -> 3)

This shows the same thing with futures that are still in the future:

scala>   val m2 = m mapValues (_.value.get.get)
java.util.NoSuchElementException: None.get

Overhear

Thanks, @som-snytt! The specific puzzler in this case is #37, as it happens ;-) – Aile

Answer 5

Just make a new future which waits for all futures in the map values , then builds a map to return.

Braithwaite

Answer 6

I would try to avoid using overengineered Scalaz based super-functional solutions (unless your project is already heavily Scalaz based and has tons of "computationally sophisticated" code; no offense on the "overengineered" remark):

// the map you have
val foo: Map[A, Future[B]] = ???

// get a Seq[Future[...]] so that we can run Future.sequence on it
val bar: Seq[Future[(A, B)]] = foo.map { case (k, v) => v.map(k -> _) }

// here you go; convert back `toMap` once it completes
Future.sequence(bar).onComplete { data =>
    // do something with data.toMap
}

However, it should be safe to assume that your map values are somehow generated from the map keys, which initially reside in a Seq such as List, and that the part of code that builds the initial Map is under your control as opposed to being sent from elsewhere. So I would personally take an even simpler/cleaner approach instead by not starting out with Map[A, Future[B]] in the first place.

def fetchAgeFromDb(name: String): Future[Int] = ???

// no foo needed anymore

// no Map at all before the future completes
val bar = personNames.map { name => fetchAgeFromDb(name).map(name -> _) }

// just as above
Future.sequence(bar).onComplete { data =>
    // do something with data.toMap
}

Tran

Answer 7

Is this solution acceptable : without an execution context this should works ...

def removeMapFuture[A, B](in: Future[Map[A, Future[B]]]) = {
  in.flatMap { k =>
    Future.sequence(k.map(l =>
      l._2.map(l._1 -> _)
    )).map {
      p => p.toMap
    }
  }
}

Cornetcy