Persisting Maps and Lists of properties as JSON in Grails

Question

EDIT: onload() method changed to afterLoad(): Otherwise objects might not be passed properly to the map.

---

I am currently using some domain classes with a lot of dynamic, complex properties, that I need to persist and update regularly.

I keep these in a Map structure for each class since this makes it easy for referencing in my controllers etc.

However, since Grails does not seem to be able to persist complex property types like List and Map in the DB I am using the following approach to achieve this via JSON String objects:

class ClassWithComplexProperties {

  Map complexMapStructure //not persisted
  String complexMapStructureAsJSON //updated and synched with map via onload,beforeInsert,beforeUpdate


  static transients = ['complexMapStructure']

  def afterLoad() {  //was previously (wrong!): def onLoad() {
    complexMapStructure=JSON.parse(complexMapStructureAsJSON)
  }
  def beforeInsert() {
    complexMapStructureAsJSON= complexMapStructure as JSON
  }
  def beforeUpdate() {
    complexMapStructureAsJSON= complexMapStructure as JSON
  }
  static constraints = {    
    complexMapStructureAsJSON( maxSize:20000)
  }
}

This works well as long I am only loading data from the DB, but I run into trouble when I want to save back my changes to the DB. E.g. when I do the following

/* 1. Load the json String, e.g. complexMapStructureAsJSON="""{
   data1:[[1,2],[3,4]],//A complex structure of nested integer lists    
   data1:[[5,6]] //Another one
    }""" :
*/
ClassWithComplexProperties c=ClassWithComplexProperties.get(1)

// 2. Change a value deep in the map: 
c.complexMapStructure.data1[0][0]=7

// 3. Try to save:

c.save(flush:true)

This will usually not work, since, I guess(?), GORM will ignore the save() request due to the fact that the map itself is transient, and no changes are found in the persisted properties.

I can make it work as intended if I hack step 3 above and change it to:

// 3.Alternative save:
complexMapStructureAsJSON="" //creating a change in persisted property (which will be overwritten anyway by the beforeUpdate closure)
c.save(flush:true)

To me this is not a very elegant handling of my problem. The questions:

1. Is there a simpler approach to persist my complex, dynamic map data?
2. If I need to do it the way I currently do, is there a way to avoid the hack in step 3 ?

Valaree

For option 2, have you tried using the beforeValidate event instead of beforeInsert and beforeUpdate? – Flying

That did the trick, and answered Q2. Plus one from me! – Valaree

OK, I'll write it up as a real answer! – Flying

See also my followup question: stackoverflow.com/questions/25749101 It turns out that the transient Map needs to be declared bindable: true in the constraints, to be able to receive data from form POST and from the map constructor, while the JSON field is best declared bindable: false. – Feme

Answer 1

For option 2, you can use the beforeValidate event instead of beforeInsert and beforeUpdate events to ensure that the change propagates correctly.

class ClassWithComplexProperties {

  Map complexMapStructure //not persisted
  String complexMapStructureAsJSON //updated and synched with map via onload,beforeInsert,beforeUpdate


  static transients = ['complexMapStructure']

  def onLoad() {
    complexMapStructure=JSON.parse(complexMapStructureAsJSON)
  }

// >>>>>>>>>>>>>>
  def beforeValidate() {
    complexMapStructureAsJSON= complexMapStructure as JSON
  }
// >>>>>>>>>>>>>>

  static constraints = {    
    complexMapStructureAsJSON( maxSize:20000)
  }
}

Flying

Thanks OverZealous. Will keep the question open, since I would still like to see if someone can come up with a solution to Q1. – Valaree

I think there should be afterLoad instead of onLoad. – Dayan

Answer 2

I of course do not know much about the application you are building, but it won't hurt to look up alternate data storage models particularly NOSQL databases. Grails has got some support for them too.

Raybin

I agree that this kind of data could be a great candidate for NOSQL storage. However, since we would like to keep all of our data persisted in the same database, I'm afraid it will not be an option in this case. – Valaree

Answer 3

Is there a simpler approach to persist my complex, dynamic map data?

Grails can persist List and Map out of the box, you don't need to write complex conversion code and abuse Json.

Example for Map:

class ClassWithComplexProperties {
    Map<String, String> properties    
}

def props = new ClassWithComplexProperties()
props.properties = ["foo" : "bar"]
props.save()

Example for List:

class ClassWithComplexProperties {
    List<String> properties
    static hasMany = [properties: String]
}

def props = new ClassWithComplexProperties()
props.properties = ["foo", "bar"]
props.save()

I think this is much easier and cleaner way how to deal with it.

Fogg

Yes, nowadays it can persist List and Maps properly, but it creates all sorts of intermediate tables on the DB, which sometimes is not what is wanted. Thanks though. – Feme

Yes, intermediate tables are side effect of this solution. If you want to avoid them and have the data in one column, look at my answer here: https://mcmap.net/q/901403/-grails-setting-transient-fields-in-the-map-constructor . But also mind that if you'll need to change the data from many threads, you need to solve synchronization, otherwise you will likely run into OptimisticLockException. – Fogg

Answer 4

In response to

Is there a simpler approach to persist my complex, dynamic map data?

Grails can persist Sets, Lists and Maps to the database. That may be a simpler approach than dealing with JSON conversions. To have the map persisted to the database you need to include it in the hasMany property.

Map complexMapStructure
static hasMany = [complexMapStructure: dynamicComplexPropertyObject]

The documentation suggests that using a Bag may be more efficient.

Barbirolli