How to deal with inconsistent mobx-state-tree snapshots?
Asked Answered
F

2

5

I am writing an electron app and I keep all my app data inside a single MST tree. Now I noticed that every now and then you run into a situation where data becomes inconsistent (missing reference object etc). While this can happen with any type of database, I see a particular problem with MST:

Since we have a single tree which is deserialised on start of the app and then used as a single snapshot, a single inconsistency will cause the whole application to fail. Not a single piece of data will be available to my application.

Any hints on how to deal with that?

More information

Currently I create a snapshot everytime the tree changes (onSnapshot) and save it in localStorage. So a error-usecase would be: create mst object -> create reference in some other part of the tree -> delete mst object -> onSnapshot gets triggered -> corrupt tree gets persisted. Reloading the application won't help, cause tree is persisted in corrupt state.

Fluorescent answered 3/9, 2019 at 8:5 Comment(3)
That actually sounds like a serious issue. If you can create reproducible codesandbox, you should perhaps open a ticket on github.Egidio
How did this go, did you figure out what was causing inconsistency? Or how do you deal with it? Any followup might be helpful for us here as well.Egidio
I haven't implemented it but I am planning to validate my store every 10 seconds and save it ONLY when the validation succeeds.Fluorescent
K
5

In order to avoid not consistent incoming data, I'm adding defaults in my models. For example

const collectionModel = types.model({
  type: types.optional(types.literal('collections'), 'collections'),
  preview: types.optional(
    types.model({
      products: types.array(SelectableProduct)
    }),
    {}
  ),
  data: types.optional(types.model({
    items: 24,
    perRow: 4,
    global: types.optional(EvergreenQuery, {}),
    curated: types.array(EvergreenItemSettings)
  }), {})
})

This will allow me to create an instance of collectionModel from an empty object

collection1 = collectionModel.create({})

When you are using references make sure to use safeReference From the docs

 * `types.safeReference` - A safe reference is like a standard reference, except that it accepts the undefined value by default
 * and automatically sets itself to undefined (when the parent is a model) / removes itself from arrays and maps
 * when the reference it is pointing to gets detached/destroyed.
 *
 * Strictly speaking it is a `types.maybe(types.reference(X))` with a customized `onInvalidate` option.

So if you are removing a node that is referenced somewhere else in the store, then that reference will get set to undefined. From my experience, broken references are especially difficult to debug.

I like that mobx-state-tree forces me to have a defined structure. This makes me think of the structure before writing the logic which later simplifies writing the logic.

A Hacky solution

A hack that you could do is before saving the snapshot instantiate a new model. If it succeeds then save the snapshot, if not skip it.

const MyModel = types.model({})

onSnapshot(myModelInstance, s => {
  try {
    const testModel = MyModel.create(s)
    if (localStorage) {
      // here you can save the snapshot because you know for sure it won't break

      localStorage.setItem('snap', JSON.stringify(s))
    }
  } catch(e) {
    // log or something

    // OR
    console.log(
      'snapshot failed because of',
       difference(s, JSON.parse(localStorage.getItem('snap'))
    )
  }
})



// this methos does a deep difference between two objects
export const difference = <T>(orObject: object, orBase: object): T => {
  function changes(object: any, base: any): any {
    return _.transform(object, function(
      result: any,
      value: any,
      key: any
    ): any {
      if (!_.isEqual(value, base[key])) {
        result[key] =
          _.isObject(value) && _.isObject(base[key])
            ? changes(value, base[key])
            : value;
      }
    });
  }

  return changes(orObject, orBase);
};

The diff method is very helpfull because it will make it easier to figure out what is the cause of the crash. In this way, the localStorage will have only valid snapshots and any invalid snapshot will log the cause of the issues.

Kitty answered 11/9, 2019 at 10:2 Comment(0)
C
1

Where is the data for the deserialisation coming from ? I'm not that familiar with electron but I imagine you store it locally (by snapshotting the mst tree) in between app sessions.

  1. My first hunch would be to look what happens when you serialise it? Maybe it'd be a good idea to validate the snapshot before saving it (I imagine on app close?) ?

  2. Is the inconsistency 'consistent'? What I mean is - is it the same part of the tree that causes it ? Maybe split the tree - serialise -> deserialise multiple snapshots of different parts instead of 1 big thing.

  3. While I do use mst, I don't use snapshotting, not at whole tree lvl at least, check if the tree gets frozen when the snapshot is created .. maybe, unlikely I think, changes are still done on the tree while the snapshot is being written.

Chunk answered 3/9, 2019 at 8:49 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.