Core Data nested managed object contexts and frequent deadlocks / freezes

Question

I have a problem that is almost identical to the problem described by this person here, but it hasn't get answered:

http://www.cocoabuilder.com/archive/cocoa/312683-core-data-nested-managed-object-contexts-and-frequent-deadlocks.html#312683

Here is the problem:

I have a parent MOC setup with NSPrivateQueueConcurrencyType and a persistent store coordinator set, it has a child MOC setup with NSMainQueueConcurrencyType. The idea being most of the long hard work and saves can be done on the private MOC freeing the main thread from blocking the UI. Unfortunately I seem to be running into a couple of situations that cause deadlocks.

If the child MOC (on the main thread) is performing a fetch with NSFetchedResultsController the parent context is sent an -executeFetchRequest: it can create a deadlock. Both operations are done within the context of a performBlock: for their respective MOCs although the docs seem to indicate that using a main thread concurrency type MOC on the main thread without performBlock: is fine.

It appears that the private queue is waiting on the PSCs lock which the child context on the main thread has already locked. It appears that the child context (while holding the PSCs lock) is trying to dispatch_sync to the parent context and thus they are both waiting for each other.

Is PriveQueue -> MainQueue a supported configuration? It seems most people still have the parent context on the main thread.

The main thread looks like this:

> #0    0x960f6c5e in semaphore_wait_trap ()
> #1    0x04956bb5 in _dispatch_thread_semaphore_wait ()
> #2    0x04955c8f in _dispatch_barrier_sync_f_slow ()
> #3    0x04955dea in dispatch_barrier_sync_f ()
> #4    0x01797de5 in _perform ()
> #5    0x01798547 in -[NSManagedObjectContext(_NestedContextSupport) newValuesForObjectWithID:withContext:error:] ()
> #6    0x0176416b in _PFFaultHandlerLookupRow ()
> #7    0x01763f97 in -[NSFaultHandler fulfillFault:withContext:forIndex:] ()
> #8    0x01763b75 in _PF_FulfillDeferredFault ()
> #9    0x017639f2 in _sharedIMPL_pvfk_core ()
> #10    0x017681a0 in _pvfk_11 ()
> #11    0x0001b322 in -[FBUser sectionName] at /Users/mlink/Code/x/x/FBUser.m:62
> #12    0x011a8813 in _NSGetUsingKeyValueGetter ()
> #13    0x017a0652 in -[NSManagedObject valueForKey:] ()
> #14    0x011ab8d5 in -[NSObject(NSKeyValueCoding) valueForKeyPath:] ()
> #15    0x01851f72 in -[NSFetchedResultsController(PrivateMethods) _sectionNameForObject:] ()
> #16    0x01853af6 in -[NSFetchedResultsController(PrivateMethods) _computeSectionInfo:error:] ()
> #17    0x01850ea6 in -[NSFetchedResultsController performFetch:] ()
> #18    0x0003a4fc in __62-[SYFriendsTableViewController updateFetchedResultsController]_block_invoke_0 ()
> #19    0x01797af3 in developerSubmittedBlockToNSManagedObjectContextPerform ()
> #20    0x049554f0 in _dispatch_main_queue_callback_4CF ()
> #21    0x01b3e833 in __CFRunLoopRun ()
> #22    0x01b3ddb4 in CFRunLoopRunSpecific ()
> #23    0x01b3dccb in CFRunLoopRunInMode ()
> #24    0x023d6879 in GSEventRunModal ()
> #25    0x023d693e in GSEventRun ()
> #26    0x0089aa9b in UIApplicationMain ()
> #27    0x00002656 in main at /Users/mlink/Code/x/x/main.mm:16

the private queue stack looks like this:

#0    0x960f8876 in __psynch_mutexwait ()
#1    0x97e9e6af in pthread_mutex_lock ()
#2    0x0172ec22 in -[_PFLock lock] ()
#3    0x0172ebfa in -[NSPersistentStoreCoordinator lock] ()
#4    0x01746a8c in -[NSManagedObjectContext(_NSInternalAdditions) lockObjectStore] ()
#5    0x01745030 in -[NSManagedObjectContext executeFetchRequest:error:] ()
#6    0x0009d49f in -[NSManagedObjectContext(Additions) executeFetchRequest:] at /Users/mlink/Code/objc/C/C/NSManagedObjectContext+Additions.m:44
#7    0x0002177f in +[FBUser usersForFbids:inManagedObjectContext:] at /Users/mlink/Code/x/x/FBUser.m:435
#8    0x00021fc0 in __77+[FBUser updateUserFromGraphValues:inManagedObjectContext:completionHandler:]_block_invoke_0 at /Users/mlink/Code/x/x/FBUser.m:461
#9    0x0180f9f3 in developerSubmittedBlockToNSManagedObjectContextPerform_privateasync ()
#10    0x04954ecf in _dispatch_queue_drain ()
#11    0x04954d28 in _dispatch_queue_invoke ()
#12    0x049544af in _dispatch_worker_thread2 ()
#13    0x97ea1b24 in _pthread_wqthread ()
#14    0x97ea36fe in start_wqthread ()

He also writes this:

I'm starting to think the problem is with NSFetchedResultsController which is always stuck at performFetch: when these deadlocks occur. Most of the time it will be stuck trying to fault in an object as a result of asking for it's section name. As a test I tried to reproduce what the FRC does and performed the executeFetchRequest: and then iterated through the results asking each object for it's section name. And this doesn't cause a deadlock. If I leave the FRC to do performFetch: after I do my test it will still deadlock there. I'm 99% sure that the FRC has a synchronization problem with nested contexts.

Question: Does anybody know why this problem occurs? Do you know how to solve it? Is this a bug?

Seagraves

Do you send -executeFetchRequest to the parent context? Why? – Cosmopolitan

Because the background thread (background/parent MOC) is syncing the data towards a server, I'm doing a fetch request to find out what objects that need to sync. – Seagraves

+1 same issue. For nested contexts, I am using magicalrecord main and root saving contexts. Before calling FRC performFetch, I started a sync request to the server. The code freezes at performFetch. – Monafo

@thejaz, may I ask how you get your stack trace text? I know how to produce the stack and view it in Xcode. However I couldn't copy and paste it elsewhere. – Hoitytoity

Answer 1

I just read this SO posting where fabrice truillot de chambrier recommends not to use nested contexts at present. He gives a reference to the article Core Data Growing Pains .

From that article:

NSFetchedResultsController deadlocks

You never want your application to deadlock. With NSFetchedResultsController and nested contexts, it’s pretty easy to do. Using the same UIManagedDocument setup described above, executing fetch requests in the private queue context while using NSFetchedResultsController with the main queue context will likely deadlock. If you start both at about the same time it happens with almost 100% consistency. NSFetchedResultsController is probably acquiring a lock that it shouldn’t be. This has been reported as fixed for an upcoming release of iOS.

radar://11861499 fixed in an upcoming release

This seems to describe exactly your issue.

Cosmopolitan

Thanks a lot! Many of my recent problems and suspicions are confirmed by the article you are linking to! – Seagraves

You are welcome. That article is really informative. We should upvote fabrice for giving the reference. – Cosmopolitan

Upvoted! I will go back to regular contexts with persistent stores and hope that all freezes and deadlocks disappear. – Seagraves

Joerg: I haven't done thorough tests, but my locking issues persist in iOS 5.1 but are resolved in iOS 6.0. – Alius

i still see this locking issue in 6.0 and 6.1, but not as often. – Strophanthin

This was definitely fixed in 6. I shipped a production application that had to support iOS 5-7, this issue existed only when running on 5 (and I had to put in some 5-specific workarounds). – Degradation

The exact same issue in iOS 8.1. Regression? – Decile

"This has been reported as fixed for an upcoming release of iOS." - really, iOS 9 now and this bug is still in iOS – Becalm

I'm still seeing this exact issue. Is there any known workaround? – Nollie

How amazing that we are on iOS 15.3 and this issue still there. – Sensillum

Answer 2

For my iOS 6 app, I have the same concurrency setup as the OP - a parent MOC using a private queue and a child MOC on the main thread. I also have an NSFetchedResultsController which uses the child MOC to update a UITableViewController. Both these MOCs are initialized in the AppDelegate and are to be used across the app. The AppDelegate has two methods, savePrivateThreadMOCToCoreData and saveMainThreadMOCToCoreData, to persist changes to CD. At launch, I dispatch a coredata initializer on a private queue as follows. The idea is to immediately drop the user into the table view and allow the initializer to update core data in the background.

    dispatch_async(private_queue,^{
        [CoreDataInitializer initialize];
    });

Initially, when savePrivateThreadMOCToCoreData was doing saves in a -performBlock, I was seeing the same psynch_mutex deadlocks described in "Core Data Growing Pains" linked above. I also saw crashes if I tried reading data into the TableVC while the save was in progress.

    Collection <__NSCFSet: 0x7d8ea90> was mutated while being enumerated.

To overcome these, I switched to doing saves using -performBlockAndWait. I stopped seeing deadlocks and crashes but it didn't feel right to make the UI wait for saves. Finally, I removed all calls to -performBlock* and used a plain vanilla [privateMOC save:&error] and just like that, all my issues disappeared. The fetched results controller reads partially saved data cleanly and updates the table, no more deadlocks or "mutated while being enumerated" errors.

I suspect -performBlock* is supposed to be used by other threads, ones which didn't create the MOC in question, to request operations on it. Since both my private and main thread MOCs belong to the app delegate, saves on the private MOC should not use -performBlock*.

It's probably relevant that though my build environment is iOS 6, my base deployment target SDK iOS 5.0. Seems others aren't seeing this issue with iOS 6 any more.

Hereinto

I suspected the same thing but got no proof. – Amil

Answer 3

It happens to me because the parents are set up with NSMainQueueConcurencyType

To solve that I make the managedobjectcontext for mainQueue to be a child. I called reset everytime I want to load stuff to ensure that data on mainQueue is the same with the parent. It often isn't.

Amil

Answer 4

I too got a crash related to developerSubmittedBlockToNSManagedObjectContextPerform.

In my case, consider the following method calling pattern:

[privatecontext performBlock:^{
    A(CDManager.privatecontext);
}];

where: A(CDManager.privateContext) calls B() B() calls C() C() calls D()

and: method A() and method C() contains some Core Data operations. A() already have the knowledge to which context to work on, but A() doesnt inform B() about the context and so C() too doesnt have any info about on which context to work on, so C() works on default context (main). and this causes the crash due to data inconsistently in db.

fix: all the methods which are to work on db operations are parametrized with the context they are to work on, except D() since it need not work on db operation, like:

A(context) calls B(context) B(context) calls C(context) C(context) calls D()

Dominion

Answer 5

I solved the exact same problem with deadlocks caused by simultaneous fetching from two threads (BG executed fired fetchRequest, MAIN one performed fetch of NSFRC). The solution is to create a new context for the long running sync operation. It has no parent context, it has concurrency type NSPrivateQueueConcurrencyType and it directly linked with a common PSC. After all long running work is done within this context in background, I save it and merge it with rest parallel contexts stack by using mergeChangesFromContextDidSaveNotification routine.

A great solution is implemented in Magical Record 3. See more info here: https://mcmap.net/q/469917/-nsfetchedresultscontroller-feeding-table-view-while-background-update-of-same-persistent-store-causes-deadlock.

Decile

Answer 6

The idea being most of the long hard work and saves can be done on the private MOC

How do you implement that idea? Do you use something like this:

- (void)doSomethingWithDocument:(UIManagedDocument *)document
{
    NSManagedObjectContext *parent = document.managedObjectContext.parentContext;
        [parent performBlock:^{
            /* 
               Long and expensive tasks.. 
               execute fetch request on parent context
               download from remote server               
            */
            // save document
        }];
}

I did above and got deadlock too. Then I tried not to touch the backing queue of parent context. Instead, I use plain and simple GCD to do the downloading stuff, and manipulate core data in child context (on main queue). It works fine. In this way parent context seems useless.. But at least it doesn't cause deadlock..

- (void)doSomethingWithDocument:(UIManagedDocument *)document
{
    dispatch_queue_t fetchQ = dispatch_queue_create("Flickr fetcher", NULL);
    dispatch_async(fetchQ, ^{
        // download from remote server        
        // perform in the NSMOC's safe thread (main thread)
        [document.managedObjectContext performBlock:^{ 
            // execute fetch request on parent context
            // save document  
        }];
    });
    dispatch_release(fetchQ);
}

Hoitytoity

document.managedObjectContext.parentContext perform block is a bad idea. All Tableview related stuffs fails. For example you get this error: Assertion failure in [UITableView _endCellAnimationsWithContext: – Raynata

That's actually a tableview datasource problem rather than a core data problem. The state transitions you're doing don't make sense to the tableview state engine. This is often because the data source isn't implemented correctly (for example, in the empty state returning 0 sections, then something inserts a section... boom). – Degradation

Answer 7

I just wanted to chime in and agree completely with avoiding nested contexts. I've been working in iOS 7 with nested contexts (main queue child and private queue parent) and NSFetchedResultsControllers and had an impossible to fix deadlock issue. I switched over to using independent MOCs and save notifications and the issue disappeared.

If anyone needs a quick guide for how to change their code over, this page has code that's ready to go (just ignore the nested context recommendation):

http://www.cocoanetics.com/2012/07/multi-context-coredata/

Morehouse