Core Data: parent context blocks child

Question

I'm doing some background processing in an app with core data. The background processing is done on a child managedObjectContext. Context initialization:

appDelegate = UIApplication.sharedApplication().delegate as! AppDelegate

// the moc in appDelegate is created with .MainQueueConcurrencyType
mainThreadMOC = appDelegate.managedObjectContext!
backgroundMOC = NSManagedObjectContext(concurrencyType:NSManagedObjectContextConcurrencyType.PrivateQueueConcurrencyType)
backgroundMOC?.parentContext = mainThreadMOC

Background processing is done in the following method:

// download all new transaction log entries
func syncItems() {

... set up the query object for parse

let moc = CoreDataStore.sharedInstance.backgroundMOC

// perform download
moc?.performBlock( {
    self.runQuery(query)   // Download stuff und do some core data work
    })
}

The debugger shows that all work inside the block is indeed in a background thread.

When I call this function from the main thread and immediately block the main thread (for test purpose) with a lengthy core data operation, I see that the background thread stops and only continues execution when the main thread is idle.

// this is called from a view controller in the main thread

syncItems() // should start to work in background
for i in 0...200 {
    // do some core data work in main thread
}
// syncItems starts to work after the blocking for-loop ends.

Why is that happening?

Foreordain

child context is not intended for background processing – Grantee

Answer 1

Don’t use a parent-child context setup.

Parent-child context are not a good approach for just about anything. Just use a simple stack: two contexts with one shared persistent store coordinator.

Parent-child contexts just add a lot of confusion without giving you much of anything. It’s a pretty misunderstood concept. I wish people like mzarra would f***ing stop advocating that setup. It’s a disservice to the community.

If your background context is a child context of your main context, you will have to lock both the main and the background context, whenever the background context needs to save. This blocks the UI. And you’ll have to lock the UI a 2nd time to propagate those changes from the UI into the PSC. If you use a background context, you will have to merge changes into the main context, but you'll only do work for those objects that are currently registered with that context. If you added new objects, or updated / deleted objects that are not currently registered (referenced) that’s basically a no-op.

Scrapbook

The problem is its being recommended for helping with multi-threading whereas it was designed as a scratchpad for making atomic changes to the main context and can be thrown away if there's an error, which is it useful for. – Grantee

Yes. As a scratch-pad they can be a reasonable approach. But people that recommend it as a way to do multi-threading simply do not know what they're talking about. – Scrapbook

100% agree and shockingly some high profile developers have made this mistake! I might look back and watch the WWDC video and see if there are any clues to why this massive mistake happened. And btw, sorry I got 'is' and 'it' the wrong way around in my first comment. – Grantee

@DanielEggert Interesting point of view. I'm curious then what would be the task of each of the two contexts that you propose and (how) do they interact? What would be their concurrencyType? – View

One context uses MainQueueConcurrencyType, one uses PrivateQueueConcurrencyType. You merge changes by listening to did-save-notifications and merging them with mergeChangesFromContextDidSaveNotification(). That's described many places, including out book: objc.io/books/core-data – Scrapbook

"Simple" stack is basically the best in terms of simplicity to performance ratio(if there can be such metric :D). If performance is not enough, use two separate PSC connected to one store. BTW, highly recommend the objc.io CoreData book. Has a ton of great information. Thank you, Daniel and Florian. :) – Ethbinium

Reading Daniel's again answer after a while and he is 100% correct about people advocating using parent-child for background are doing a disservice to the community. Perfect example of API abuse. – Grantee

I believe UIManagedDocument added some confusion because it uses the same parent-child context structure. I've seen its usage as CoreData stack for non-document apps(actually I've never worked with document-based ones). – Ethbinium

Apple's documentation is one of the source of the parent/child setup: developer.apple.com/library/content/documentation/Cocoa/… – Humbug

Hi @DanielEggert, I was thinking the same like it is complexe and confusing but not giving me much. you said merging changes between context by listening to notifications. is it good practice to just use refreshAllObjects() when even a background context finishes saving ? – Lorri

Answer 2

Regarding parent contexts, unfortunately the last time I checked the documentation was fairly spartan, and online you will find that nearly every single person has inverted the flow. There was one WWDC session on Core Data back in 2011 or so where everything was made clear, and if you look at the API carefully, it will start to make sense.

The child is NOT the background context - it is the main context. The child is the context you are generally interacting with. The parent is the background context.

The child pushes changes to its parent. That is why the parent has a persistentStoreCoordinator, but the child instead has a parentContext (it does NOT need a persistentStoreCoordinator).

So the child is your main context, on the main (UI) queue. It saves changes up into its parent. This happens in memory (fast - leaving the UI as responsive as possible).

The parent context then saves its changes into the persist store via its persistentStoreCoordinator. That is why the parent is on a private queue.

lazy var managedObjectContext: NSManagedObjectContext = {
    let parentContext = NSManagedObjectContext(concurrencyType: .PrivateQueueConcurrencyType)
    parentContext.persistentStoreCoordinator = self.persistentStoreCoordinator

    let managedObjectContext = NSManagedObjectContext(concurrencyType: .MainQueueConcurrencyType)
    managedObjectContext.parentContext = parentContext
    return managedObjectContext
}()

There are other optimizations, such as setting the .undoManager to nil, but this general architecture works flawlessly for backgrounding saves.

You probably also want to put in a save method that receives a completion block/closure, immediately save on your child queue (which only saves into the parent, as mentioned), then call the parentContext's performBlock method in which you will have it save (on its private queue) into the underlying persistent store (comparatively slow, but now non-blocking), and then call your completion block/closure (which you either already set up using GCD to run back on the main queue, or else you call back to the main queue from there in the parent's performBlock method.

Everything falls into place perfectly fine when you don't invert the architecture. I'm not sure how it got started online, but it's a nearly universal mistake.

Best of luck.

Trivalent

This is not the right approach because if the child is on the main thread, when it saves to the parent, it has to wait on the background queue to finish saving anyway, because after all the user cannot continue if their operation was not successful for whatever reason, e.g. an error in the data they input or a merge error. This negates any performance advantage you attempted and have actually just included unnecessary complexity and memory overhead. Do not use parent/child for threading, it simply was not designed for it. It was designed as a scratch pad for throw away editing of objects. – Grantee

@malhal: I did not get ur point exactly for the main purpose of parent/child? Can you give an example on what do you mean by throw away editing of objects? – Hedgehop

Think of it as "atomicity of changes". For example, imagine creating a new Category object in the first view controller, then a new Product on the second view controller, and a final save button. If you had just used the main context, and say the Product entry was cancelled, or it failed on validation, then you would be left with an unsaved Category in the context. If you had created a child context and passed that along the view controllers, then at the save stage if it fails or is cancelled then the context is just thrown away and the main context remains in a consistent state. – Grantee

@malhal: I really appreciate your comments/explanation. But I still have a question: is there a way to prevent a context from blocking the access to the store/db file (say while it's saving a HUGE amount of data) , so another context accessing the same store can always fetch without waiting? When the data numbers in 10s of thousand and is hierachic, the statement in this anwer "This happens in memory (fast...)" no longer holds. Thanks. – Humbug

@Humbug yes use a background context – Grantee

Thanks. But not in a simple parent-child setup, right? I had asked this: #42849562 – Humbug

Answer 3

When you say in the comment "do some core data work in main thread" is that work accessing mainThreadMOC?

It sounds like perhaps the main thread work is locking something that runQuery needs to access.

Try changing the testing busy-work that blocks the main thread to something that doesn't access Core Data (NSThread.sleepForTimeInterval should do) and see if that lets the background runQuery work.

If that's the problem you'll need to refactor the main thread work into something that doesn't block whatever is happening in runQuery.

Opera

Answer 4

I suspect your problem is twofold, in that, even though you are doing for test purposes, your

lengthy core data operation

on the main thread is blocking the UI, and by your definition your backgroundMOC?.parentContext = mainThreadMOC.

I'd recommend creating a more robust structure for your multiple NSManagedObjectContexts.

I recommend you follow the steps outlined in this answer.

In addition, you can create an additional MOC specifically to manage your runQuery...

discreteMOC = NSManagedObjectContext(concurrencyType:NSManagedObjectContextConcurrencyType.PrivateQueueConcurrencyType)
discreteMOC?.parentContext = backgroundMOC //MOCprivate in my previous answer. 

Hope this helps.

Klagenfurt

How would you comment on Daniel's answer above? In a child(main-queue)/parent(private queue) setup, if the parent is busy processing(eg. saving), the child can only wait when trying fetch from its parent store(the parent). – Humbug

@DanielEggert may be well versed in the subject but his answer demonstrates primarily opinion. I don't experience the UI blocks he mentions. I'd like to experiment with his suggestion to remove parent-child contexts in my code, but until then I can't comment on whether it's good advice or not. Each implementation of Core Data has different needs - mine seem to benefit from using parent-child contexts, but whether they're necessary? I would ask why he's so upset about Marcus Zarra's approach? Seems to come from an honest premise. It's helped me enormously, I've learned a lot about Core Data. – Klagenfurt

Thanks for the rspns:-) In my project the parent-background-child-main(plus temporary scratch pad contexts) setup does hit the blocking issue. When saving(deleting) in the parent context (10s of thousands objects), the child can only wait when it needs to fetch. That causes the beach ball rolling for a few seconds(really annoying). I "disguise" the issue by delaying the saving to the app's becoming inactive, but that has introduced another issue: since the objs are not deleted yet, they still appear in the fetch result. So I had to do some ugly hacking, but it's not working 100% as expected. – Humbug

@Humbug that's cool! Honestly it took me a long time to get my head around parent-child context model (PCCM) but once I did it made sense. I still don't claim to understand threading very well. For my implementations, what is critical when using PCCM, is placing regularly performed saves to memory (fast) on the main context thread (child) and save to disk (slow) on a private context thread (parent) obviously with the PSC hooked to the private parent. PCCM means no messy notification or merge coding. The private context will manage data interface and ensure interactions with PSC don't block UI. – Klagenfurt

In that setting, the private context's activity may block the UI if the main-queue context wants to fetch from the parent at the same time...since the parent happens to be busy. Currently I'm trying to solve the "hiding those will-be-deleted-but-not-yet objects" problem. It sounds like an awful idea but I need to try different approaches to alleviating the blocking (at least superficially). – Humbug

Have you posted a question? If you have or will, I can perhaps answer in more detail and also dump my standard implementation in to demonstrate how it works for me. – Klagenfurt

Hi @Klagenfurt Just posted: #42849562 Thank you very much! – Humbug