global identifiers? - iCloud + Core Data + Ensembles - duplicates when deleting objects

Question

I am trying to implement iCloud sync in my Core Data app. I am not that pro in programming and this is really an advanced topic I learned... I found that Core Data sync Framework "Ensembles" by Drew McCormack. It seems to make iCloud Sync much easier.

I integrated it in my App and syncing does work quite well as long as I add new objects to my Core Data model. But when I delete an object, it creates duplicates. And then duplicates from duplicates. I ended up having the same Entry (object) like 3-4 times...

Why is that? What am I doing wrong? I did some research and my guess is that global identifiers could solve this?

What are global identifiers? My guess is that they help to avoid duplicates!? But how do I set this? I really have no idea, did a lot of research but couldn´t find an answer to that.

Thanks for help!

Update: Thanks for help! I read the readme and the book, but since i am beginner not everything is clear to me.

I think I understand the use of global identifiers in Ensembles now, but I don´t know if I´m doing it correctly.

If I understand it right, I have to assign an identifier to each object. I can do this by storing it in an attribute. This identifier can be anything as long as it is unique and a NSString?

In my app the user can store different things, let´s say name, text, title, date and so on. The app is based on the Master-Detail-View template in Xcode and uses Core Data. My Core Data model has only a single entity with some attributes, most are strings and a NSDate. No relationships or anything. If the user hits "+" a new object is created and I store the things the user enters in the attributes.

What I did to add global identifiers is to add a new attribute that stores it. So when a new object is created i do

/// I did find that to use as identifier !?

NSString *taskUniqueStringKey = newManagedObject.objectID.URIRepresentation.absoluteString;

/// and store it in the attribute.

[newManagedObject setValue:taskUniqueStringKey forKey:@"coreDataObjectID"]; 

Then i use this:

- (NSArray *)persistentStoreEnsemble:(CDEPersistentStoreEnsemble *)ensemble globalIdentifiersForManagedObjects:(NSArray *)objects
{

return [objects valueForKeyPath:@"coreDataObjectID"];;

}

This seems to work for me. But am I doing it right? Is this the right place to assign a global identifier? I have no awakeFromInsert !?

If this is working, I got the next problem. My app is already live and older entries that the user saved before the update will be missing the global identifier. What can I do about that? I thought what I already got and what is unique and the only thing I can think of is an attribute that saves [NSDate date] when the object is created.

I was trying to use this but I failed because Ensembles will only accept NSString and not NSDate!? Can I use this date attribute, is this unique enough and working as gloabl identifier? And if yes, could you please give me code example in how to convert this from date to string?

Syncing with Ensembles works quite good. No duplicates anymore, you can just switch off iCloud and the entries stay and switch it on again and it syncs like it should without loosing locally stored objects or so. Ensembles is really cool! I am seeing some minor strange behaviors like sometimes sync takes long, sometimes it´s really quick and if I edit things in a short time period on two different devices it gets a bit messed up like an object that I just deleted reappears. But I guess that´s normal? If I take some time between using the app on the different devices everything works fine.

Do I understand it right, there is only that one method to call for sync:

- (void)syncWithCompletion:(void(^)(void))completion
{
if (self.ensemble.isMerging) return;


if (!self.ensemble.isLeeched) {
    [self.ensemble leechPersistentStoreWithCompletion:^(NSError *error) {
        if (error) NSLog(@"Error in leech: %@", error);

        if (completion) completion();
    }];
}
else {
    [self.ensemble mergeWithCompletion:^(NSError *error) {

        if (completion) completion();
    }];
}

and you just call it if needed? There is nothing else like doing merge without leeching before, or a method like "this is the actual status - save it like it is now" ?

There are different points in the app where you want to sync. On app start and when terminating will be a good point. In my app there are two points where I should sync I guess: when adding an object and save it to Core Data and when I save changes to the object. I could also provide a button like "sync now". Is this a good approach and do I always just call

[self syncWithCompletion:NULL];

Another question that came up. Can I exclude objects from sync with Ensembles? My app loads tutorial entries as objects once on first app start. I don´t want to sync them if that´s possible somehow?

Thanks a lot for your help! If I could help you with anything like localizing in german or so let me know ! ;)

Protrusion

I'm trying Ensembles but haven't been able to make it sync at all - do you mind sharing some setup code? Thanks! – Ghetto

I really did not do anything else than following the instructions for a basic setup. Integrated the franework, set up the ensembles with the identifier in diddinishlaunching and then called syncwithcompletion whenever needed. Could also use these addobserver methods. Take a look at the example projects. This basic project included all you need. It s very simple. – Protrusion

What i found out while testing - i needed to use a custom icloud container instead of the default one to make sync across devices work. If i used the default container my data was backed up in icloud but did not sync across different devises. Don t know if it s a must, don t really understand the difference of the containers, but for me it only worked with a custom container. – Protrusion

Thanks for the quick reply! And interesting fact about the custom container. Did you follow the Simple Sync or Idiomatic? – Ghetto

Simple sync. I only wanted icloud and not dropbox and so on... So it has everything i needed. – Protrusion

It s everything in the appdelegate.m just set up the ensembles in diddinishlaunchingwithoptions and call syncwithcompletion. You need global identifiers for the core data objects to not get duplicates, especially when deleting objects. That s all. The rest is optional. These addobserver methods can be used to update screen when sync completed and when you call sync is optional, what best fits your needs... – Protrusion

It s everything in the appdelegate.m just set up the ensembles in diddinishlaunchingwithoptions and call syncwithcompletion. You need global identifiers for the core data objects to not get duplicates, especially when deleting objects. That s all. The rest is optional. These addobserver methods can be used to update screen when sync completed. when you call sync is optional, what best fits your needs. In most cases you want call sync on app start and finish probably like in the example... In my app i call sync when user changes the core data model, when adding, changing or removing an object. – Protrusion

Finally got simple sync working - thanks for the help! I currently have iCloud + Core Data so I need migration and it's kind of complicated. If you happen to have similar situation let me know :) – Ghetto

What do you mean by core data + icloud need migration? – Protrusion

Before Ensembles, I've already set up Core Data with iCloud syncing mechanism and pushed the app to store with some users, so I can't start with Ensembles from scratch but instead need to get data from iCloud Core Data first, then switch to Ensembles, which makes it troublesome – Ghetto

I had to move images stored in documents directory to core data for an update using ensembles. Did that by copying all on first launch of the update with a for-loop – Protrusion

I see so you basically did manual copy of the objects. How did you switch up psc? If the user deletes the app and re-install it, how do you check if they still need a switch to Ensembles? Could you send sample code to [email protected]? Thanks so much for all the help! – Ghetto

Or even better, make a github and let people collaborate to make an optimal approach to migrate core data stuff into a syncing mechanism ;) – Ghetto

I m not at home for the weekend, no mac no code here... Will do when home... I am github noob, can t do anything there... Don t know if my copy approach is that what you need but will tell you when i m home – Protrusion

Great - I can set up the github if you like :) – Ghetto

i don t think i can really help you with that. I did not change to Ensembles. I did not have iCloud before. I only had to move images that were not stored in core data before to core data. I did a copy with a for loop once in didfinishlaunchungwithoptions. used nsuserdefaults to ensure this copy action will execute only once - when launching update for the first time... – Protrusion

if(![[[NSUserDefaults standardUserDefaults] valueForKey:@"coreDataMigrationDocDir"] isEqualToString:@"yes"] && [[NSUserDefaults standardUserDefaults] integerForKey:@"ApplaunchCount"] > 1) { – Protrusion

for (NSManagedObject *object in objectsArray) { "copy action code" – Protrusion

[[NSUserDefaults standardUserDefaults] setValue:@"yes" forKey:@"coreDataMigrationDocDir"]; [[NSUserDefaults standardUserDefaults] synchronize]; } – Protrusion

Answer 1

Yes, this is almost certainly due to not setting up global identifiers for your objects, or at least not doing it properly.

When you leech your ensemble, the local persistent store is imported into the sync data. Without global identifiers, Ensembles will assign random ids to your objects, so it can track them across devices.

Duplicates arise when you leech a second device that has the same data. Ensembles has no way to know that the data represents the same logical objects as on the other device, so it again assigns random ids. Effectively, it treats the objects on each device as being completely independent, so that all end up in your data set after syncing.

The solution is global identifiers. By implementing a CDEPersistentStoreEnsemble delegate method, you can provide Ensembles with global ids, which it can use to identify which objects on different devices belong together.

What should you use for global ids? Often, just a UUID, though for singleton like objects you will just want to pick an id.

You can initialize them in awakeFromInsert. You can store the global ids in attributes on your entities. (Note that if you are migrating an existing app, you will want to check with a fetch if the global ids have been generated BEFORE you try to leech the store for syncing.)

More details are in the README on GitHub and in the book at leanpub.

Update

To answer your update questions:

Yes, an identifier just has to be a string, and immutable. It should not change once assigned.

The NSManagedObjectID is not a very good global identifier, in that it will be different on different devices. We really want something that is global across devices.

If you are starting from scratch, using NSUUID is a good approach. Just create a unique id, and store it in the object.

If you have an existing app, and it has been syncing via another mechanism, you need to come up with a way to provide the same global identifiers on each device. One way to do that is mash up the object properties in some way. Usually that will give you a pretty-close-to-unique value, and it will be good enough for the transition.

As an example, you do a quick fetch, and discover that your objects don't yet have global ids. You go through the objects, and set the global ids to a string comprised of creationDate + text. (You could even shorten this by taking a hash, but it probably isn't that important.) After this initial 'migration' to global identifiers, you would just use UUIDs for any newly created objects.

Note that you don't have to use awakeFromInsert. That is simply a convenient place to put it. As long as you assign the global identifier before saving the object you should be fine.

The easiest way to get a string from an NSDate is to call the description method, but another way would be to get a double using timeIntervalSince1970, and turning that into a string. (Be careful with dates as unique identifiers on their own: often objects created together will have the same creation date.)

You are correct about how you should do a sync: you can simply call syncWithCompletion:.

To answer the question about excluding objects: You can't exclude individual objects, mainly because it could become tricky when those objects have relationships to synced objects. You can handle these objects in one of two ways:

1. Put them in a separate persistent store, and add that store to the same persistent store coordinator.
2. Sync the objects, but give them global ids manually, so that the objects are treated the same on each device. Eg. You could just give global ids as 'Sample1', 'Sample2', etc.

Mammon

One quick additional question: is Ensembles still working after adding attributes to an entity and doing lightweight migration? I did just this, I am testing and it doesn t seem to sync anymore and I thought it s because of the new core data model... could this be ? thanks ! – Protrusion

It should work with lightweight migrations. Note that syncing will pause until all devices have the new model, at which point they should integrate all the data. Each device continues to capture saved changes, so there should be no data loss. – Mammon

thanks for answer, it s exactly like you said... I had different models and it did not sync, when i updated to same version all synced again... thanks – Protrusion

Answer 2

To integrate Drew's answer, I guess the two steps are the following.

1 Implement CDEPersistentStoreEnsemble delegate method (see README)

- (NSArray *)persistentStoreEnsemble:(CDEPersistentStoreEnsemble *)ensemble 
    globalIdentifiersForManagedObjects:(NSArray *)objects {
    return [objects valueForKeyPath:@"yourUniqueIdentifier"];
}

2 Generate the unique identifier for a NSManagedObject subclass

- (void)awakeFromInsert {
    [super awakeFromInsert];

    if (!self.yourUniqueIdentifier) {
        self.yourUniqueIdentifier = [[NSUUID UUID] UUIDString];
    }
}

In awakeFromInsert you can initialize special default property values, like for example an identifier.

The check is necessary, for example, when you have parent-child contexts. Otherwise you are overwriting the identifier previously set. See Why is awakeFromInsert called twice?.

Clisthenes

I know this sounds silly but how would you write up if (!self.yourUniqueIdentifier) { self.yourUniqueIdentifier = [[NSUUID UUID] UUIDString]; } in swift ? – Dispassion

@JKSDEV let uniqueIdentifier = NSUUID().UUIDString – Landgrabber