Using Core Data Concurrently and Reliably

Question

I'm building my first iOS app, which in theory should be pretty straightforward but I'm having difficulty making it sufficiently bulletproof for me to feel confident submitting it to the App Store.

Briefly, the main screen has a table view, upon selecting a row it segues to another table view that displays information relevant for the selected row in a master-detail fashion. The underlying data is retrieved as JSON data from a web service once a day and then cached in a Core Data store. The data previous to that day is deleted to stop the SQLite database file from growing indefinitely. All data persistence operations are performed using Core Data, with an NSFetchedResultsController underpinning the detail table view.

The problem I am seeing is that if you switch quickly between the master and detail screens several times whilst fresh data is being retrieved, parsed and saved, the app freezes or crashes completely. There seems to be some sort of race condition, maybe due to Core Data importing data in the background whilst the main thread is trying to perform a fetch, but I'm speculating. I've had trouble capturing any meaningful crash information, usually it's a SIGSEGV deep in the Core Data stack.

The table below shows the actual order of events that happen when the detail table view controller is loaded:

Main Thread                          Background Thread
viewDidLoad

                                     Get JSON data (using AFNetworking)

Create child NSManagedObjectContext (MOC)

                                     Parse JSON data
                                     Insert managed objects in child MOC
                                     Save child MOC
                                     Post import completion notification

Receive import completion notification
Save parent MOC
Perform fetch and reload table view

                                     Delete old managed objects in child MOC
                                     Save child MOC
                                     Post deletion completion notification

Receive deletion completion notification
Save parent MOC

Once the AFNetworking completion block is triggered when the JSON data has arrived, a nested NSManagedObjectContext is created and passed to an "importer" object that parses the JSON data and saves the objects to the Core Data store. The importer executes using the new performBlock method introduced in iOS 5:

NSManagedObjectContext *child = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
    [child setParentContext:self.managedObjectContext];        
    [child performBlock:^{
        // Create importer instance, passing it the child MOC...
    }];

The importer object observes its own MOC's NSManagedObjectContextDidSaveNotification and then posts its own notification which is observed by the detail table view controller. When this notification is posted the table view controller performs a save on its own (parent) MOC.

I use the same basic pattern with a "deleter" object for deleting the old data after the new data for the day has been imported. This occurs asynchronously after the new data has been fetched by the fetched results controller and the detail table view has been reloaded.

One thing I am not doing is observing any merge notifications or locking any of the managed object contexts or the persistent store coordinator. Is this something I should be doing? I'm a bit unsure how to architect this all correctly so would appreciate any advice.

Crenelation

Answer 1

Pre-iOS 5, we've usually had two NSManagedObjectContexts: one for the main thread, one for a background thread. The background thread can load or delete data and then save. The resulting NSManagedObjectContextDidSaveNotification was then passed (as you're doing) to the main thread. We called mergeChangesFromManagedObjectContextDidSaveNotification: to bring those in to the main thread context. This has worked well for us.

One important aspect of this is that the save: on the background thread blocks until after the mergeChangesFromManagedObjectContextDidSaveNotification: finishes running on the main thread (because we call mergeChanges... from the listener to that notification). This ensures that the main thread managed object context sees those changes. I don't know if you need to do this if you have a parent-child relationship, but you did in the old model to avoid various kinds of trouble.

I'm not sure what the advantage of having a parent-child relationship between the two contexts is. It seems from your description that the final save to disk happens on the main thread, which probably isn't ideal for performance reasons. (Especially if you might be deleting a large amount of data; the major cost for deletion in our apps has always happened during final save to disk.)

What code are you running when the controllers appear/disappear that could be causing core data trouble? What kinds of stack traces are you seeing the crash on?

Baines

Thanks. I'm actually not using mergeChangesFromManagedObjectContextDidSaveNotification: because in normal usage the new data shows up perfectly fine without it. However, I have used it in a previous build and had the same crashes when switching quickly between the two screens. – Crenelation

Answer 2

Just an architectural idea:

With your stated data refresh pattern (once a day, FULL cycle of data deleted and added), I would actually be motivated to create a new persistent store each day (i.e. named for the calendar date), and then in the completion notification, have the table view setup a new fetchedresultscontroller associated with the new store (and likely a new MOC), and refresh using that. Then the app can (elsewhere, perhaps also triggered by that notification) completely destroy the "old" data store. This technique decouples the update processing from the data store that the app is currently using, and the "switch" to the new data might be considered dramatically more atomic, since the change happens simply be starting to point to the new data instead of hoping you aren't catching the store in an inconsistent state while new data is being written (but is not yet complete).

Obviously I have left some details out, but I tend to think that much data being changed while being used should be re-architected to reduce the likelihood of the kind of crash you are experiencing.

Happy to discuss further...

Ahmadahmar

I like this idea, I like it a lot. It also has the advantage that a file system delete of the SQLite file will be a lot quicker than having Core Data delete the managed objects in the object graph, although of course actually the deletion performance doesn't matter because a different persistent store will be used anyway. I'm going to give this approach a go this weekend. – Crenelation

Answer 3

The main issue I've had with multi-threaded core data is inadvertently accessing a managed object in a thread/queue other than the one it was created in.

I've found a good debugging tool is add NSAsserts to check that to managed objects created in your main managed object context are only used there, and ones created in a background context aren't used in the main context.

This will involve subclassing NSManagedObjectContext and NSManagedObject:

• Add a iVar to the MOC subclass and assign to it the queue it was created on.
• Your MO subclass should check the current queue is the same as its MOC's queue property.

It's only a few lines of code, but long term can prevent you making errors that are hard to track down otherwise.

Manhandle

Thanks, but I'm not sure how to get the MOC's queue since it's being instantiated using initWithConcurrencyType:NSPrivateQueueConcurrencyType which means that it creates its own private dispatch queue. – Crenelation

How about dispatch_get_current_queue inside your performBlock: – Manhandle

That queue is private; trying to access it directly is something the Apple folks call out as a bad idea. – Baines

@JesseRusak true, but I'm not suggesting you do anything with it - just use it to compare against the queue that a managed object's moc was created on/with. – Manhandle

Answer 4

NSFetchedResultsController has been proven to be a bit sensitive to massive deletes so that is where I would start digging first.

My initial question is, how are the re-fetch and reload of tableview related to the start of delete operation. Is there a chance that the deletion block will save child MOC while the NSFetchedResultsController is still fetching or no?

Is it possible that when you switch from detail view to master and then back to detail view there will be multiple concurrent background tasks running? Or are you retrieving all the data from the web service at once not only that relevant to a particular row?

One alternative to make this more robust is to use a pattern similar to what UIManagedDocument uses:

Instead of using a parent MOC as Main Thread concurrency type, UIManagedDocument actually creates the main MOC as private queue and makes the child MOC available to you use on the main thread. The benefit here is that all the I/O goes on in the background and saves to the parent MOC does not interfere with the child MOC at all until child MOC is explicitly made know about them. That's because save commits changes from child to parent and not the other way around.

So if you did your deletes on a parent queue which is private, that would not end up in the NSFetchedResultsController scope at all. And since it is old data, that is actually the preferred way.

An alternative I offer is to use three contexts:

Main MOC (NSPrivateQueueConcurrencyType)

• Responsible for persistent store and deletion of old data.

Child MOC A (NSMainQueueConcurrencyType)

• Responsible for anything UI related and NSFetchedResultsController

Child MOC B (NSPrivateQueueConcurrencyType, child of Child MOC A)

• Responsible for inserting new data and committing it up to Child MOC A when done.

Weywadt