Cleaner way to update nested structures

Question

Say I have got following two case classes:

case class Address(street: String, city: String, state: String, zipCode: Int)
case class Person(firstName: String, lastName: String, address: Address)

and the following instance of Person class:

val raj = Person("Raj", "Shekhar", Address("M Gandhi Marg", 
                                           "Mumbai", 
                                           "Maharashtra", 
                                           411342))

Now if I want to update zipCode of raj then I will have to do:

val updatedRaj = raj.copy(address = raj.address.copy(zipCode = raj.address.zipCode + 1))

With more levels of nesting this gets even more uglier. Is there a cleaner way (something like Clojure's update-in) to update such nested structures?

Audrieaudris

I assume you want to preserve immutabilty, otherwise, just stick a var in front of Persons' address declaration. – Hauteloire

@GClaramunt: Yes, I want to preserve the immutability. – Audrieaudris

Answer 1

Zippers

Huet's Zipper provides convenient traversal and 'mutation' of an immutable data structure. Scalaz provides Zippers for Stream (scalaz.Zipper), and Tree (scalaz.TreeLoc). It turns out that the structure of the zipper is automatically derivable from the original data structure, in a manner that resembles symbolic differentiation of an algebraic expression.

But how does this help you with your Scala case classes? Well, Lukas Rytz recently prototyped an extension to scalac that would automatically create zippers for annotated case classes. I'll reproduce his example here:

scala> @zip case class Pacman(lives: Int = 3, superMode: Boolean = false) 
scala> @zip case class Game(state: String = "pause", pacman: Pacman = Pacman()) 
scala> val g = Game() 
g: Game = Game("pause",Pacman(3,false))

// Changing the game state to "run" is simple using the copy method:
scala> val g1 = g.copy(state = "run") 
g1: Game = Game("run",Pacman(3,false))

// However, changing pacman's super mode is much more cumbersome (and it gets worse for deeper structures):
scala> val g2 = g1.copy(pacman = g1.pacman.copy(superMode = true))
g2: Game = Game("run",Pacman(3,true))

// Using the compiler-generated location classes this gets much easier: 
scala> val g3 = g1.loc.pacman.superMode set true
g3: Game = Game("run",Pacman(3,true)

So the community needs to persuade the Scala team that this effort should be continued and integrated into the compiler.

Incidentally, Lukas recently published a version of Pacman, user programmable through a DSL. Doesn't look like he used the modified compiler, though, as I can't see any @zip annotations.

Tree Rewriting

In other circumstances, you might like to apply some transformation across the entire data structure, according to some strategy (top-down, bottom-up), and based on rules that match against the value at some point in the structure. The classical example is transforming an AST for a language, perhaps to evaluate, simplify, or collect information. Kiama supports Rewriting, see the examples in RewriterTests, and watch this video. Here's a snippet to whet your appetite:

// Test expression
val e = Mul (Num (1), Add (Sub (Var ("hello"), Num (2)), Var ("harold")))

// Increment every double
val incint = everywheretd (rule { case d : Double => d + 1 })
val r1 = Mul (Num (2), Add (Sub (Var ("hello"), Num (3)), Var ("harold")))
expect (r1) (rewrite (incint) (e))

Note that Kiama steps outside the type system to achieve this.

Beauharnais

For those looking for the commit. Here it is: github.com/soundrabbit/scala/commit/… (I think..) – Gyrostabilizer

Hey, where are the lenses? – Salable

I've just encountered this problem and the @zip idea sounds really fantastic, maybe it should even be taken so far that all case classes has it? Why isn't this implemented? Lenses are nice but with large and many classes/case classes it's just boilerplate if you just want a setter and nothing fancy like an incrementer. – Wirework

Answer 2

Funny that no one added lenses, since they were MADE for this kind of stuff. So, here is a CS background paper on it, here is a blog which touch briefly on lenses use in Scala, here is a lenses implementation for Scalaz and here is some code using it, which looks surprisingly like your question. And, to cut down on boiler plate, here's a plugin that generate Scalaz lenses for case classes.

For bonus points, here's another S.O. question which touches on lenses, and a paper by Tony Morris.

The big deal about lenses is that they are composable. So they are a bit cumbersome at first, but they keep gaining ground the more you use them. Also, they are great for testability, since you only need to test individual lenses, and can take for granted their composition.

So, based on an implementation provided at the end of this answer, here's how you'd do it with lenses. First, declare lenses to change a zip code in an address, and an address in a person:

val addressZipCodeLens = Lens(
    get = (_: Address).zipCode,
    set = (addr: Address, zipCode: Int) => addr.copy(zipCode = zipCode))

val personAddressLens = Lens(
    get = (_: Person).address, 
    set = (p: Person, addr: Address) => p.copy(address = addr))

Now, compose them to get a lens that changes zipcode in a person:

val personZipCodeLens = personAddressLens andThen addressZipCodeLens

Finally, use that lens to change raj:

val updatedRaj = personZipCodeLens.set(raj, personZipCodeLens.get(raj) + 1)

Or, using some syntactic sugar:

val updatedRaj = personZipCodeLens.set(raj, personZipCodeLens(raj) + 1)

Or even:

val updatedRaj = personZipCodeLens.mod(raj, zip => zip + 1)

Here's the simple implementation, taken from Scalaz, used for this example:

case class Lens[A,B](get: A => B, set: (A,B) => A) extends Function1[A,B] with Immutable {
  def apply(whole: A): B   = get(whole)
  def updated(whole: A, part: B): A = set(whole, part) // like on immutable maps
  def mod(a: A, f: B => B) = set(a, f(this(a)))
  def compose[C](that: Lens[C,A]) = Lens[C,B](
    c => this(that(c)),
    (c, b) => that.mod(c, set(_, b))
  )
  def andThen[C](that: Lens[B,C]) = that compose this
}

Salable

You might want to update this answer with a description of Gerolf Seitz's lenses plugin. – Audrieaudris

@Audrieaudris Sure. Link? I wasn't aware of such plugin. – Salable

The code personZipCodeLens.set(raj, personZipCodeLens.get(raj) + 1) is the same as personZipCodeLens mod (raj, _ + 1) – Tristram

@Tristram mod isn't a primitive for lenses, though. – Salable

Tony Morris has written a great paper on the subject. I think you should link it in your answer. – Audrieaudris

also see: shapeless case-classes lenses – Punchinello

Good to remember that: (_: Address).zipCode is just shortcut of (a: Address) => a.zipCode. It's easier to read for not scala-pro. – Yokel

the composition of these Lenses lets to you chain into a deep struction. popular libraries/macros let you combine lenses in an 'additive' fashion (I don't know the correct term) such that val combined = lensX ~ lensY lets you set both X and Y at the same time in the structure even though X and Y are in different parts of the structure. What is the equivalent with the Lens case class in this answer? My first guess is to use the pimp pattern to add arbitrary set methods but that seems very boilerplate and inelegant. – Schism

Answer 3

Useful tools to use Lenses:

Just want to add that the Macrocosm and Rillit projects, based on Scala 2.10 macros, provides Dynamic Lens Creation.

---

Using Rillit:

case class Email(user: String, domain: String)
case class Contact(email: Email, web: String)
case class Person(name: String, contact: Contact)

val person = Person(
  name = "Aki Saarinen",
  contact = Contact(
    email = Email("aki", "akisaarinen.fi"),
    web   = "http://akisaarinen.fi"
  )
)

scala> Lenser[Person].contact.email.user.set(person, "john")
res1: Person = Person(Aki Saarinen,Contact(Email(john,akisaarinen.fi),http://akisaarinen.fi))

---

Using Macrocosm:

This even works for case classes defined in the current compile run.

case class Person(name: String, age: Int)

val p = Person("brett", 21)

scala> lens[Person].name._1(p)
res1: String = brett

scala> lens[Person].name._2(p, "bill")
res2: Person = Person(bill,21)

scala> lens[Person].namexx(()) // Compilation error

Newsmonger

You probably missed Rillit which is even better. :-) github.com/akisaarinen/rillit – Audrieaudris

Btw I edited my answer to include Rillit but I don't really understand why Rillit is better, they seems to provide the same functionality in the same verboseness at first sight @Audrieaudris – Newsmonger

@SebastienLorber Fun fact: Rillit is Finnish and means Lenses :) – Antonantone

Both Macrocosm and Rillit seem not be updated in the last 4 years. – Fetation

Answer 4

I've been looking around for what Scala library that has the nicest syntax and the best functionality and one library not mentioned here is monocle which for me has been really good. An example follows:

import monocle.Macro._
import monocle.syntax._

case class A(s: String)
case class B(a: A)

val aLens = mkLens[B, A]("a")
val sLens = aLens |-> mkLens[A, String]("s")

//Usage
val b = B(A("hi"))
val newB = b |-> sLens set("goodbye") // gives B(A("goodbye"))

These are very nice and there are many ways to combine the lenses. Scalaz for example demands a lot of boilerplate and this compiles quick and runs great.

To use them in your project just add this to your dependencies:

resolvers ++= Seq(
  "Sonatype OSS Releases"  at "http://oss.sonatype.org/content/repositories/releases/",
  "Sonatype OSS Snapshots" at "http://oss.sonatype.org/content/repositories/snapshots/"
)

val scalaVersion   = "2.11.0" // or "2.10.4"
val libraryVersion = "0.4.0"  // or "0.5-SNAPSHOT"

libraryDependencies ++= Seq(
  "com.github.julien-truffaut"  %%  "monocle-core"    % libraryVersion,
  "com.github.julien-truffaut"  %%  "monocle-generic" % libraryVersion,
  "com.github.julien-truffaut"  %%  "monocle-macro"   % libraryVersion,       // since 0.4.0
  "com.github.julien-truffaut"  %%  "monocle-law"     % libraryVersion % test // since 0.4.0
)

Wirework

Answer 5

Shapeless does the trick:

"com.chuusai" % "shapeless_2.11" % "2.0.0"

with:

case class Address(street: String, city: String, state: String, zipCode: Int)
case class Person(firstName: String, lastName: String, address: Address)

object LensSpec {
      import shapeless._
      val zipLens = lens[Person] >> 'address >> 'zipCode  
      val surnameLens = lens[Person] >> 'firstName
      val surnameZipLens = surnameLens ~ zipLens
}

class LensSpec extends WordSpecLike with Matchers {
  import LensSpec._
  "Shapless Lens" should {
    "do the trick" in {

      // given some values to recreate
      val raj = Person("Raj", "Shekhar", Address("M Gandhi Marg",
        "Mumbai",
        "Maharashtra",
        411342))
      val updatedRaj = raj.copy(address = raj.address.copy(zipCode = raj.address.zipCode + 1))

      // when we use a lens
      val lensUpdatedRaj = zipLens.set(raj)(raj.address.zipCode + 1)

      // then it matches the explicit copy
      assert(lensUpdatedRaj == updatedRaj)
    }

    "better yet chain them together as a template of values to set" in {

      // given some values to recreate
      val raj = Person("Raj", "Shekhar", Address("M Gandhi Marg",
        "Mumbai",
        "Maharashtra",
        411342))

      val updatedRaj = raj.copy(firstName="Rajendra", address = raj.address.copy(zipCode = raj.address.zipCode + 1))

      // when we use a compound lens
      val lensUpdatedRaj = surnameZipLens.set(raj)("Rajendra", raj.address.zipCode+1)

      // then it matches the explicit copy
      assert(lensUpdatedRaj == updatedRaj)
    }
  }
}

Note that whilst some other answers here let you compose lenses to go deeper into a given structure these shapless lenses (and other libraries/macros) let you combine two unrelated lenses such that you can make lens that sets an arbitrary number of parameters into arbitrary positions in your structure. For complex data structures that additional composition is very helpful.

Schism

Note that I eventually ended up using the Lens code in Daniel C. Sobral's answer and so avoided adding an external dependency. – Schism

Answer 6

Due to their composable nature, lenses provide a very nice solution to the problem of heavily nested structures. However with a low level of nesting, I sometimes feel lenses are a bit too much, and I don't want to introduce the whole lenses approach if there is only few places with nested updates. For sake of completeness, here is a very simple/pragmatic solution for this case:

What I do is to simply write a few modify... helper functions in the top level structure, which deal with the ugly nested copy. For instance:

case class Person(firstName: String, lastName: String, address: Address) {
  def modifyZipCode(modifier: Int => Int) = 
    this.copy(address = address.copy(zipCode = modifier(address.zipCode)))
}

My main goal (simplifying the update on client side) is achieved:

val updatedRaj = raj.modifyZipCode(_ => 41).modifyZipCode(_ + 1)

Creating the full set of modify helpers is obviously annoying. But for internal stuff it is often okay to just create them the first time you try to modify a certain nested field.

Electrolyte

Answer 7

Short answer

Use QuickLens:

val updatedRaj = raj.modify(_.address.zipCode).using(_ + 1)

Works with scala 2.11+ and 3+.

Long answer

Perhaps QuickLens matches your question better. QuickLens uses macro's to convert an IDE friendly expression into something that is close to the original copy statement.

Given the two example case classes:

case class Address(street: String, city: String, state: String, zipCode: Int)
case class Person(firstName: String, lastName: String, address: Address)

and the instance of Person class:

val raj = Person("Raj", "Shekhar", Address("M Gandhi Marg", 
                                           "Mumbai", 
                                           "Maharashtra", 
                                           411342))

you can update zipCode of raj with:

import com.softwaremill.quicklens._
val updatedRaj = raj.modify(_.address.zipCode).using(_ + 1)

Fetation