Why DispatchQueue.main.async is required when using CoreData, NSFetchedResultsController and Diffable Data Source

Question

When dealing with CoreData, NSFetchedResultsController and Diffable Data Source, I always notice that I need to apply DispatchQueue.main.async.

For instance,

Before applying DispatchQueue.main.async

extension ViewController: NSFetchedResultsControllerDelegate {
    func controller(_ fetchedResultsController: NSFetchedResultsController<NSFetchRequestResult>, didChangeContentWith snapshotReference: NSDiffableDataSourceSnapshotReference) {
        guard let dataSource = self.dataSource else {
            return
        }
        
        var snapshot = snapshotReference as NSDiffableDataSourceSnapshot<String, NSManagedObjectID>

        dataSource.apply(snapshot, animatingDifferences: true) { [weak self] in
            guard let self = self else { return }
        }
    }
}

However, after we run performFetch in viewDidLoad, I will get the following error in dataSource.apply

'Deadlock detected: calling this method on the main queue with outstanding async updates is not permitted and will deadlock. Please always submit updates either always on the main queue or always off the main queue

I can "resolve" the problem by using the following

After applying DispatchQueue.main.async

extension ViewController: NSFetchedResultsControllerDelegate {
    func controller(_ fetchedResultsController: NSFetchedResultsController<NSFetchRequestResult>, didChangeContentWith snapshotReference: NSDiffableDataSourceSnapshotReference) {
        DispatchQueue.main.async { [weak self] in
            guard let self = self else { return }
            
            guard let dataSource = self.dataSource else {
                return
            }
            
            var snapshot = snapshotReference as NSDiffableDataSourceSnapshot<String, NSManagedObjectID>

            dataSource.apply(snapshot, animatingDifferences: true) { [weak self] in
                guard let self = self else { return }
            }
        }
    }
}

---

Things work fine after that.

But, we are puzzled on why DispatchQueue.main.async is ever required because

1. performFetch is run in main thread.
2. Callback didChangeContentWith is run in main thread.
3. NSFetchedResultsController is using main CoreData context, not background context.

Hence, we cannot understand why we are getting runtime error if DispatchQueue.main.async is not used.

Do you have idea, why DispatchQueue.main.async is required when using CoreData, NSFetchedResultsController and Diffable Data Source?

The following are our detailed code snippet.

CoreDataStack.swift

import CoreData

class CoreDataStack {
    public static let INSTANCE = CoreDataStack()
    
    private init() {
    }
    
    lazy var persistentContainer: NSPersistentContainer = {
        let container = NSPersistentContainer(name: "xxx")
        
        container.loadPersistentStores(completionHandler: { (storeDescription, error) in
            if let error = error as NSError? {
                // This is a serious fatal error. We will just simply terminate the app, rather than using error_log.
                fatalError("Unresolved error \(error), \(error.userInfo)")
            }
        })
        
        // So that when backgroundContext write to persistent store, container.viewContext will retrieve update from
        // persistent store.
        container.viewContext.automaticallyMergesChangesFromParent = true
        
        // TODO: Not sure these are required...
        //
        //container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        //container.viewContext.undoManager = nil
        //container.viewContext.shouldDeleteInaccessibleFaults = true
        
        return container
    }()
    
    lazy var backgroundContext: NSManagedObjectContext = {
        let backgroundContext = persistentContainer.newBackgroundContext()

        // TODO: Not sure these are required...
        //
        backgroundContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        //backgroundContext.undoManager = nil
        
        return backgroundContext
    }()
    
    // https://www.avanderlee.com/swift/nsbatchdeleterequest-core-data/
    func mergeChanges(_ changes: [AnyHashable : Any]) {
        
        // TODO:
        //
        // (1) Should this method called from persistentContainer.viewContext, or backgroundContext?
        // (2) Should we include backgroundContext in the into: array?
        
        NSManagedObjectContext.mergeChanges(
            fromRemoteContextSave: changes,
            into: [persistentContainer.viewContext, backgroundContext]
        )
    }
}

NoteViewController.swift

class NoteViewController: UIViewController {
    
    override func viewDidLoad() {
        super.viewDidLoad()

        ...
        initDataSource()
        initNSTabInfoProvider()
    }

    
    private func initNSTabInfoProvider() {
        self.nsTabInfoProvider = NSTabInfoProvider(self)
        
        // Trigger performFetch
        _ = self.nsTabInfoProvider.fetchedResultsController
    }

    private func initDataSource() {
        let dataSource = DataSource(
            collectionView: tabCollectionView,
            cellProvider: { [weak self] (collectionView, indexPath, objectID) -> UICollectionViewCell? in
                
                guard let self = self else { return nil }
                
                ...
            }
        )
        
        self.dataSource = dataSource
    }

NSTabInfoProvider.swift

import Foundation
import CoreData

// We are using https://github.com/yccheok/earthquakes-WWDC20 as gold reference.
class NSTabInfoProvider {
    
    weak var fetchedResultsControllerDelegate: NSFetchedResultsControllerDelegate?
    
    lazy var fetchedResultsController: NSFetchedResultsController<NSTabInfo> = {
        
        let fetchRequest = NSTabInfo.fetchSortedRequest()
        
        // Create a fetched results controller and set its fetch request, context, and delegate.
        let controller = NSFetchedResultsController(
            fetchRequest: fetchRequest,
            managedObjectContext: CoreDataStack.INSTANCE.persistentContainer.viewContext,
            sectionNameKeyPath: nil,
            cacheName: nil
        )
        
        controller.delegate = fetchedResultsControllerDelegate
        
        // Perform the fetch.
        do {
            try controller.performFetch()
        } catch {
            error_log(error)
        }
        
        return controller
    }()
    
    var nsTabInfos: [NSTabInfo]? {
        return fetchedResultsController.fetchedObjects
    }
    
    init(_ fetchedResultsControllerDelegate: NSFetchedResultsControllerDelegate) {
        self.fetchedResultsControllerDelegate = fetchedResultsControllerDelegate
    }
    
    func getNSTabInfo(_ indexPath: IndexPath) -> NSTabInfo? {
        guard let sections = self.fetchedResultsController.sections else { return nil }
        return sections[indexPath.section].objects?[indexPath.item] as? NSTabInfo
    }
}

Kite

Answer 1

I have found out the root cause of the problem.

This is due to my insufficient understanding on the lazy initialised variable.

Problematic code

class NSTabInfoProvider {
    lazy var fetchedResultsController: NSFetchedResultsController<NSTabInfo> = {
        
        let fetchRequest = NSTabInfo.fetchSortedRequest()
        
        // Create a fetched results controller and set its fetch request, context, and delegate.
        let controller = NSFetchedResultsController(
            fetchRequest: fetchRequest,
            managedObjectContext: CoreDataStack.INSTANCE.persistentContainer.viewContext,
            sectionNameKeyPath: nil,
            cacheName: nil
        )
        
        controller.delegate = fetchedResultsControllerDelegate
        
        // Perform the fetch.
        do {
            try controller.performFetch()
        } catch {
            error_log(error)
        }
        
        return controller
    }()
}

self.nsTabInfoProvider = NSTabInfoProvider(self)
// Trigger performFetch
_ = self.nsTabInfoProvider.fetchedResultsController

1. It is wrong to trigger performFetch in lazy variable initialisation.
2. Because that will trigger callback.
3. Callback might try to access NSTabInfoProvider's fetchedResultsController.
4. But NSTabInfoProvider's fetchedResultsController is NOT fully initialised, as the code is not yet return from the lazy variable initialisation scope.

---

Fixed code

The solution would be

class NSTabInfoProvider {
    lazy var fetchedResultsController: NSFetchedResultsController<NSTabInfo> = {
        
        let fetchRequest = NSTabInfo.fetchSortedRequest()
        
        // Create a fetched results controller and set its fetch request, context, and delegate.
        let controller = NSFetchedResultsController(
            fetchRequest: fetchRequest,
            managedObjectContext: CoreDataStack.INSTANCE.persistentContainer.viewContext,
            sectionNameKeyPath: nil,
            cacheName: nil
        )
        
        controller.delegate = fetchedResultsControllerDelegate
        
        return controller
    }()

    func performFetch() {
        do {
            try self.fetchedResultsController.performFetch()
        } catch {
            error_log(error)
        }
    }
}

self.nsTabInfoProvider = NSTabInfoProvider(self)
self.nsTabInfoProvider.performFetch()

Kite

Answer 2

I think the issue is to do with the fact the models are probably added or updated using a background context

lazy var backgroundContext: NSManagedObjectContext = {
    let backgroundContext = persistentContainer.newBackgroundContext()

    // TODO: Not sure these are required...
    //
    backgroundContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
    //backgroundContext.undoManager = nil
    
    return backgroundContext
}()

This could be why you need to push everything to the main thread because in your method you are trying to update your datasource (by extension your tableview) which is a UI component, thus it needs to be on the main thread.

You can think of the main thread as a UI thread.

Norvol

Answer 3

Please note the highlighted part of the run time error.

'Deadlock detected: calling this method on the main queue with outstanding async updates is not permitted and will deadlock. Please always submit updates either always on the main queue or always off the main queue

This is clearly mentioned in the UICollectionViewDiffableDataSource.apply docs as well.

Discussion

The diffable data source computes the difference between the collection view’s current state and the new state in the applied snapshot, which is an O(n) operation, where n is the number of items in the snapshot.

You can safely call this method from a background queue, but you must do so consistently in your app. Always call this method exclusively from the main queue or from a background queue.

What do you need to do?

Check for all call sites of UICollectionViewDiffableDataSource.apply in your code and make sure those are consistently off / on main thread. You can't call this from multiple threads (one time from main, other time from some other thread etc.)

Oolite