Updating nested immutable data structures

Question

I want to update a nested, immutable data structure (I attached a small example of a hypothetical game.) And I wonder whether this can be done a little more elegantly.

Every time something inside the dungeon changes we need a new dungeon. So, I gave it a general update member. The best way to use this, that I could come up with for the general case, is to specify the processing functions for each nesting and than pass the combined function to the update member.

Then, for really common cases (like applying a map to all the monsters on a specific level), I provide extra members (Dungeon.MapMonstersOnLevel).

The whole thing works, I would just like to know, if anyone can think of better ways of doing it.

Thanks!

// types
type Monster(awake : bool) = 
    member this.Awake = awake

type Room(locked : bool, monsters : Monster list) = 
    member this.Locked = locked
    member this.Monsters = monsters

type Level(illumination : int, rooms : Room list) = 
    member this.Illumination = illumination
    member this.Rooms = rooms

type Dungeon(levels : Level list) = 
    member this.Levels = levels

    member this.Update levelFunc = 
        new Dungeon(this.Levels |> levelFunc)

    member this.MapMonstersOnLevel (f : Monster -> Monster) nLevel =
        let monsterFunc = List.map f
        let roomFunc = List.map (fun (room : Room) -> new Room(room.Locked, room.Monsters |> monsterFunc))
        let levelFunc = List.mapi (fun i (level : Level) -> if i = nLevel then new Level(level.Illumination, level.Rooms |> roomFunc) else level)
        new Dungeon(this.Levels |> levelFunc)

    member this.Print() = 
        this.Levels 
        |> List.iteri (fun i e -> 
            printfn "Level %d: Illumination %d" i e.Illumination
            e.Rooms |> List.iteri (fun i e ->
                let state = if e.Locked then "locked" else "unlocked"
                printfn "  Room %d is %s" i state
                e.Monsters |> List.iteri (fun i e ->
                    let state = if e.Awake then "awake" else "asleep"
                    printfn "    Monster %d is %s" i state)))

// generate test dungeon
let m1 = new Monster(true)
let m2 = new Monster(false)
let m3 = new Monster(true)
let m4 = new Monster(false)
let m5 = new Monster(true)
let m6 = new Monster(false)
let m7 = new Monster(true)
let m8 = new Monster(false)
let r1 = new Room(true, [ m1; m2 ])
let r2 = new Room(false, [ m3; m4 ])
let r3 = new Room(true, [ m5; m6 ])
let r4 = new Room(false, [ m7; m8 ])
let l1 = new Level(100, [ r1; r2 ])
let l2 = new Level(50, [ r3; r4 ])
let dungeon = new Dungeon([ l1; l2 ])
dungeon.Print()

// toggle wake status of all monsters
let dungeon1 = dungeon.MapMonstersOnLevel (fun m -> new Monster(not m.Awake)) 0
dungeon1.Print()

// remove monsters that are asleep which are in locked rooms on levels where illumination < 100 and unlock those rooms
let monsterFunc2 = List.filter (fun (monster : Monster) -> monster.Awake)
let roomFunc2 = List.map(fun (room : Room) -> if room.Locked then new Room(false, room.Monsters |> monsterFunc2) else room)
let levelFunc2 = List.map(fun (level : Level) -> if level.Illumination < 100 then new Level(level.Illumination, level.Rooms |> roomFunc2) else level)
let dungeon2 = dungeon.Update levelFunc2
dungeon2.Print()

Phiz

Have you considered using records or single-cased union types? – Nubbly

@RamonSnir: I try to explain myself at pad's answer below. – Phiz

Answer 1

I asked a similar question, but about haskell: Is there a Haskell idiom for updating a nested data structure?

The excellent answers mentioned a concept known as functional lenses.

---

Unfortunately, I don't know what the package is, or if it even exists, for F#.

Update: two knowledgeable F#-ists (F#-ers? F#as?) left useful links about this in comments, so I'll post them here:

• @TomasPetricek suggested FSharpX and this website describing it

• @RyanRiley gave the link for the package

It's awesome that these two guys took the time to read my answer, comment and improve it, as they're both developers of FSharpX!

---

More extraneous information: I was motivated to figure out how to do this by Clojure's assoc-in and update-in functions, which proved to me that it is possible in functional languages! Of course, Clojure's dynamic typing makes it simpler than in Haskell/F#. Haskell's solution involves templating, I believe.

Lido

There is an implementation of lenses for F# in FSharpX package. See description by Mauricio Scheffer: bugsquash.blogspot.com/2011/11/lenses-in-f.html. It looks like an interesting option, but it is still a somewhat experimental design - it isn't really a standard pattern used in F# (yet?). – Granophyre

Thanks for pointing out your question - lots of interesting links in there! – Phiz

@TomasPetricek: Thanks, I take a look at that. – Phiz

Watch FSharpx for an upcoming NuGet release that includes lenses. You can also get lenses now if you pull the source. – Heartbroken

Answer 2

Here's the same code using lenses as currently defined in FSharpx. As other answers note, it's convenient to use records here; they give you structural equality for free among other things. I also attach the corresponding lenses for the properties as static members; you can also define them in a module or as loose functions. I prefer static members here, for practical purposes it's just like a module.

open FSharpx

type Monster = {
    Awake: bool
} with 
    static member awake =
        { Get = fun (x: Monster) -> x.Awake
          Set = fun v (x: Monster) -> { x with Awake = v } }

type Room = {
    Locked: bool
    Monsters: Monster list
} with
    static member locked = 
        { Get = fun (x: Room) -> x.Locked
          Set = fun v (x: Room) -> { x with Locked = v } }
    static member monsters = 
        { Get = fun (x: Room) -> x.Monsters
          Set = fun v (x: Room) -> { x with Monsters = v } }

type Level = {
    Illumination: int
    Rooms: Room list
} with
    static member illumination = 
        { Get = fun (x: Level) -> x.Illumination
          Set = fun v (x: Level) -> { x with Illumination = v } }
    static member rooms = 
        { Get = fun (x: Level) -> x.Rooms
          Set = fun v (x: Level) -> { x with Rooms = v } }

type Dungeon = {
    Levels: Level list
} with
    static member levels =
        { Get = fun (x: Dungeon) -> x.Levels 
          Set = fun v (x: Dungeon) -> { x with Levels = v } }
    static member print (d: Dungeon) = 
        d.Levels 
        |> List.iteri (fun i e -> 
            printfn "Level %d: Illumination %d" i e.Illumination
            e.Rooms |> List.iteri (fun i e ->
                let state = if e.Locked then "locked" else "unlocked"
                printfn "  Room %d is %s" i state
                e.Monsters |> List.iteri (fun i e ->
                    let state = if e.Awake then "awake" else "asleep"
                    printfn "    Monster %d is %s" i state)))

I also define print as a static member; again it's like a function in a module, and it's more composable than an instance method (though I won't compose it here).

Now to generate the sample data. I think { Monster.Awake = true } is more desciptive than new Monster(true). If you wanted to use classes I'd name the parameter explicitly, e.g. Monster(awake: true)

// generate test dungeon
let m1 = { Monster.Awake = true }
let m2 = { Monster.Awake = false }
let m3 = { Monster.Awake = true }
let m4 = { Monster.Awake = false }
let m5 = { Monster.Awake = true }
let m6 = { Monster.Awake = false }
let m7 = { Monster.Awake = true }
let m8 = { Monster.Awake = false }

let r1 = { Room.Locked = true;  Monsters = [m1; m2] }
let r2 = { Room.Locked = false; Monsters = [m3; m4] }
let r3 = { Room.Locked = true;  Monsters = [m5; m6] }
let r4 = { Room.Locked = false; Monsters = [m7; m8] }

let l1 = { Level.Illumination = 100; Rooms = [r1; r2] }
let l2 = { Level.Illumination = 50;  Rooms = [r3; r4] }

let dungeon = { Dungeon.Levels = [l1; l2] }
Dungeon.print dungeon

Now comes the fun part: composing lenses to update the monsters for all rooms for a particular level in a dungeon:

open FSharpx.Lens.Operators

let mapMonstersOnLevel nLevel f =
    Dungeon.levels >>| Lens.forList nLevel >>| Level.rooms >>| Lens.listMap Room.monsters
    |> Lens.update (f |> List.map |> List.map)

// toggle wake status of all monsters
let dungeon1 = dungeon |> mapMonstersOnLevel 0 (Monster.awake.Update not)
Dungeon.print dungeon1

For the second dungeon I also use lenses but without lens composition. It's sort of a DSL defined by small composed functions (some of the functions are from lenses). Maybe there are lenses to express this more concisely, but I haven't figured it out.

// remove monsters that are asleep 
// which are in locked rooms on levels where illumination < 100 
// and unlock those rooms

let unlock = Room.locked.Set false
let removeAsleepMonsters = Room.monsters.Update (List.filter Monster.awake.Get)

let removeAsleepMonsters_unlock_rooms = List.mapIf Room.locked.Get (unlock >> removeAsleepMonsters)

let isLowIllumination = Level.illumination.Get >> ((>)100)
let removeAsleepMonsters_unlock_level = Level.rooms.Update removeAsleepMonsters_unlock_rooms
let removeAsleepMonsters_unlock_levels = List.mapIf isLowIllumination removeAsleepMonsters_unlock_level

let dungeon2 = dungeon |> Dungeon.levels.Update removeAsleepMonsters_unlock_levels
Dungeon.print dungeon2

I overused lenses and pointfree a bit here, partially on purpose, just to show what it could look like. Some won't like it, claiming it's not idiomatic or clear. Maybe so, but it's another tool that you can choose to use or not, depending on your context.

But more importantly, because Update is a Get followed by a function followed by a Set, this isn't as efficient as your code when it comes to processing lists: an Update in Lens.forList first gets the nth element in the list, which is an O(n) operation.

To summarize:

Pros:

• Very concise.
• Enables pointfree style.
• Code involving lenses is generally oblivious of the source type representation (it can be a class, a record, a single-case DU, a dictionary, it doesn't matter).

Cons:

• May be inefficient for some cases in current implementation.
• Due to lack of macros, requires some boilerplate.

Thanks for this example, as a result I'll be revising the current design of lenses in FSharpx and see if it can be optimized.

I committed this code to the FSharpx repository: https://github.com/fsharp/fsharpx/commit/136c763e3529abbf91ad52b8127ce11cbb3dff28

Astto

That is awesome. Thank you very much for investing this much work in my silly little example and not only answering the question, but providing a whole lot of extra things to think about. Incredibly appreciated. (I don't want to take away an already accepted answer though, so hope you're alright with a simple upvote). – Phiz

Answer 3

I don't know why you want to use classes here. I think you can leverage the power of pattern matching if you use records for holding data and keeping them minimal:

// Types
type Monster = {
    Awake: bool
    }
    with override x.ToString() =
            if x.Awake then "awake" else "asleep"
type Room = {
    Locked: bool;
    Monsters: Monster list
    }
    with override x.ToString() =
            let state = if x.Locked then "locked" else "unlocked"
            state + "\n" + (x.Monsters |> List.mapi (fun i m -> sprintf "    Monster %d is %s" i (string m)) |> String.concat "\n")

type Level = {
    Illumination : int;
    Rooms : Room list
    }
    with override x.ToString() =
              (string x.Illumination) + "\n" + (x.Rooms |> List.mapi (fun i r -> sprintf "  Room %d is %s" i (string r)) |> String.concat "\n")

type Dungeon = {
    Levels: Level list;
    }
    with override x.ToString() =
            x.Levels |> List.mapi (fun i l -> sprintf "Level %d: Illumination %s" i (string l)) |> String.concat "\n"

To me, putting functions for manipulating Dungeon inside the class is unnatural. The code looks better if you put them in a module and make use of above declarations:

/// Utility functions
let updateMonster (m: Monster) a =
    {m with Awake = a}

let updateRoom (r: Room) l monstersFunc =
    {   Locked = l; 
        Monsters = r.Monsters |> monstersFunc}    

let updateLevel (l: Level) il roomsFunc = 
    {Illumination = il; Rooms = l.Rooms |> roomsFunc}

let updateDungeon (d: Dungeon) levelsFunc =
    {d with Levels = d.Levels |> levelsFunc}


/// Update functions
let mapMonstersOnLevel (d: Dungeon) nLevel =
    let monstersFunc = List.map (fun m -> updateMonster m (not m.Awake))
    let roomsFunc = List.map (fun r -> updateRoom r r.Locked monstersFunc)
    let levelsFunc = List.mapi (fun i l -> if i = nLevel then updateLevel l l.Illumination roomsFunc else l)
    updateDungeon d levelsFunc

let removeSleptMonsters (d: Dungeon) =
    let monstersFunc = List.filter (fun m -> m.Awake)
    let roomsFunc = List.map (fun r -> if r.Locked then updateRoom r false monstersFunc else r)
    let levelsFunc = List.map (fun l -> if l.Illumination < 100 then updateLevel l l.Illumination roomsFunc else l)
    updateDungeon d levelsFunc

Then you can see manipulating these nested data structures is much easier. However, above functions still have redundancy. You can refactor more if you use lenses which come very natural with records. Check out the insightful article by Mauricio Scheffer, which is really close to this formulation.

Middleoftheroader

Thanks for working through this thing! As to why I used classes: In a more realistic application, the types tend to be more complex, with helper functions etc., and I found it easier, to provide a discoverable interface using classes and I didn't find the lack of the copy-with-syntax of records to be much of a problem. You mentioned pattern matching, but I can't find where you're using it? I will take a look into lenses, thanks for the tip! – Phiz

@Phiz : Records can have member functions too; they're just classes under the hood, after all. – Chancellorship

@ildjarn: I know, but the advantage I saw with classes, was that they give you easy private helper functions, whereas with records you'd have to put them in a private module or use signature files to hide them away. I'm still struggling with the best way to architect components, to be honest. – Phiz

@Phiz Record types can have members just as classes. The difference is that they may only have one public constructor which assigns to all the fields, which in turn must be public. – Nubbly

@Ramon Snir: But then how do you propose to keep the type's helper functions from leaking out? Make them private members? Put them in a private module? Use an .fsi? I'm curious, although maybe this matter deserves a whole new question... – Phiz

@bknotic: I prefer to use a private module since using .fsi helps to restrict access control in the assembly level. – Middleoftheroader

@RamonSnir: Actually you can have an universal access control for record fields. See msdn.microsoft.com/en-us/library/dd233188.aspx (Community Content part). – Middleoftheroader

pad: yes, I just don't like this (doesn't mean its impossible or illegal). I just prefer to do this using the .fsi file. preference. @kbnotic all solutions are essentially the same. If you have only a few, go for private members. If you have many, go for a private module. If you don't mind having two files (.fs and .fsi) go for .fsi. – Nubbly

@pad, Ramon Snir: Thanks guys, I will definitely take another look at component design trying out your suggestions. – Phiz

@Phiz F# for fun and profit is a fantastic resource for designing less tightly coupled systems through modules and functional programming. – Sealer

Answer 4

I posted a similar question about Scala about a year back. The answers mention three concepts as a solution to this problem: Zippers, Tree rewriting, and Lenses.

Cowden

Thanks for pointing out your question. This thread accumulated quite a bit of knowledge about solving the problem now! – Phiz

Answer 5

I've implemented a lens library in C# via reflection. The core of the library is this function

/// <summary>
/// Perform an immutable persistent set on a sub
/// property of the object. The object is not
/// mutated rather a copy of the object with
/// the required change is returned.
/// </summary>
/// <typeparam name="ConvertedTo">type of the target object</typeparam>
/// <typeparam name="V">type of the value to be set</typeparam>
/// <param name="This">the target object</param>
/// <param name="names">the list of property names composing the property path</param>
/// <param name="value">the value to assign to the property</param>
/// <returns>A new object with the required change implemented</returns>
private static T Set<T, V>
    (this T This, List<string> names, V value)
    where T : class, Immutable
{
    var name = names.First();
    var rest = names.Skip(1).ToList();
    if (names.Count == 1)
    {
        var copy = This.ShallowClone();
        copy.SetPrivatePropertyValue(names.First(), value);
        return copy as T;
    }
    else
    {
        var copy = This.ShallowClone();
        var subtree = copy
            .GetPrivatePropertyValue<Immutable>(name)
            .Set(rest, value);

        copy.SetPrivatePropertyValue(names.First(), subtree);
        return copy as T;
    }
}

The above function is composed using helper library into various utilities, one of which is an undo stack based on immutable persistent records. There is an overload of this function

public static Maybe<T> MaybeSet<T,V>
    (this T This, Expression<Func<T, V>> prop, V value)
    where T : class, Immutable
{
    if (!EqualityComparer<V>.Default.Equals(This.Get(prop.Compile()),value))
    {
        var names = ReactiveUI.Reflection.ExpressionToPropertyNames(prop).ToList();
        return This.Set(names, value).ToMaybe();
    }
    else
    {
        return None<T>.Default;
    }
}

which allows more natural type safe notation using LINQ expressions.

foo = foo.Set(f=>f.A.B.C, 10);

There is a lot of reflection going on in the library but the reduction in boilerplate is worth the performance hit. See the spec. I only need to tag the record as Immutable to get it to work. I don't have to provide getters and settors.

class A : Immutable
{
    public int P { get; private set; }
    public B B { get; private set; }
    public A(int p, B b)
    {
        P = p;
        B = b;
    }
}

class B : Immutable
{
    public int P { get; private set; }
    public C C { get; private set; }
    public B(int p, C c)
    {
        P = p;
        C = c;
    }
}

class C : Immutable
{
    public int P { get; private set; }
    public C(int p)
    {
        P = p;
    }
}


namespace Utils.Spec
{
    public class ImmutableObjectPatternSpec : IEnableLogger
    {
        [Fact]
        public void SetterSpec()
        {
            var a = new A
                ( p:10
                , b: new B
                    ( p: 20
                    , c : new C(30)));

            var a_ = a.Set(p => p.B.C.P, 10);

            a.Should().NotBe(a_);
            a.B.C.P.Should().Be(30);
            a_.B.C.P.Should().Be(10);
        }

        [Fact]
        public void StringListGettersShouldWork()
        {
            var a = new A
                ( p:10
                , b: new B
                    ( p: 20
                    , c : new C(30)));

            var a_ = a.Set(p => p.B.C.P, 10);

            a_.Get(p=>p.B.C.P).Should().Be(10);

        }




    }
}

Perhaps reflection based lenses would reduce boiler plate in F#. Maybe performance could be improved with caching of the accessors or maybe IL generation.

Ellette