iOS 11+ How to migrate existing Core Data to Shared App Group for use in extension?

Question

When I created an iOS 11 app using the core data template, it auto generated the following code in AppDelete.m.

synthesize persistentContainer = _persistentContainer;

- (NSPersistentContainer *)persistentContainer {
    // The persistent container for the application. This implementation creates and returns a container, having loaded the store for the application to it.
    @synchronized (self) {
        if (_persistentContainer == nil) {
            _persistentContainer = [[NSPersistentContainer alloc] initWithName:@"My_History"];
            [_persistentContainer loadPersistentStoresWithCompletionHandler:^(NSPersistentStoreDescription *storeDescription, NSError *error) {
                if (error != nil) {
                    // Replace this implementation with code to handle the error appropriately.
                    // abort() causes the application to generate a crash log and terminate. You should not use this function in a shipping application, although it may be useful during development.

                    /*
                     Typical reasons for an error here include:
                     * The parent directory does not exist, cannot be created, or disallows writing.
                     * The persistent store is not accessible, due to permissions or data protection when the device is locked.
                     * The device is out of space.
                     * The store could not be migrated to the current model version.
                     Check the error message to determine what the actual problem was.
                    */
                    NSLog(@"Unresolved error %@, %@", error, error.userInfo);
                    abort();
                }
            }];
        }
    }

    return _persistentContainer;
}

- (void)saveContext {
NSManagedObjectContext *context = self.persistentContainer.viewContext;
NSError *error = nil;
if ([context hasChanges] && ![context save:&error]) {
    // Replace this implementation with code to handle the error appropriately.
    // abort() causes the application to generate a crash log and terminate. You should not use this function in a shipping application, although it may be useful during development.
    NSLog(@"Unresolved error %@, %@", error, error.userInfo);
    abort();
}

I would like to add a Today and iMessage extension that accesses the history in the core data. From what I read, I need to migrate this data if it exists to a shared app container. How would I do that?

The code is in objective C.

I have read other questions involving this but all of them seem to be before Apple changed the way core data works to make it easier. As you can see in my code, I never specified what the data store exact file name is. Every example I saw had something like "My_History.sqllite". I don't even know if mine is a sql lite database, it was just created by that code.

Peritoneum

Answer 1

I ended up getting it doing the following. The sqlite file was actually the name of my init plus .sqlite at the end.

+ (NSPersistentContainer*) GetPersistentContainer {
    //Init the store.
    NSPersistentContainer *_persistentContainer = [[NSPersistentContainer alloc] initWithName:@"Test_App"];

    //Define the store url that is located in the shared group.
    NSURL* storeURL = [[[NSFileManager defaultManager] containerURLForSecurityApplicationGroupIdentifier:@"group.Test_App"] URLByAppendingPathComponent:@"Test_App.sqlite"];

    //Determine if we already have a store saved in the default app location.
    BOOL hasDefaultAppLocation = [[NSFileManager defaultManager] fileExistsAtPath: _persistentContainer.persistentStoreDescriptions[0].URL.path];

    //Check if the store needs migration.
    BOOL storeNeedsMigration = hasDefaultAppLocation && ![_persistentContainer.persistentStoreDescriptions[0].URL.absoluteString isEqualToString:storeURL.absoluteString];

    //Check if the store in the default location does not exist.
    if (!hasDefaultAppLocation) {
        //Create a description to use for the app group store.
        NSPersistentStoreDescription *description = [[NSPersistentStoreDescription alloc] init];

        //set the automatic properties for the store.
        description.shouldMigrateStoreAutomatically = true;
        description.shouldInferMappingModelAutomatically = true;

        //Set the url for the store.
        description.URL = storeURL;

        //Replace the coordinator store description with this description.
        _persistentContainer.persistentStoreDescriptions = [NSArray arrayWithObjects:description, nil];
    }

    //Load the store.
    [_persistentContainer loadPersistentStoresWithCompletionHandler:^(NSPersistentStoreDescription *storeDescription, NSError *error) {
        //Check that we do not have an error.
        if (error == nil) {
            //Check if we need to migrate the store.
            if (storeNeedsMigration) {
                //Create errors to track migration and deleting errors.
                NSError *migrateError;
                NSError *deleteError;

                //Store the old location URL.
                NSURL *oldStoreURL = storeDescription.URL;

                //Get the store we want to migrate.
                NSPersistentStore *store = [_persistentContainer.persistentStoreCoordinator persistentStoreForURL: oldStoreURL];

                //Set the store options.
                NSDictionary *storeOptions = @{ NSSQLitePragmasOption : @{ @"journal_mode" : @"WAL" } };

                //Migrate the store.
                NSPersistentStore *newStore = [_persistentContainer.persistentStoreCoordinator migratePersistentStore: store toURL:storeURL options:storeOptions withType:NSSQLiteStoreType error:&migrateError];

                //Check that the store was migrated.
                if (newStore && !migrateError) {
                    //Remove the old SQLLite database.
                    [[[NSFileCoordinator alloc] init] coordinateWritingItemAtURL: oldStoreURL options: NSFileCoordinatorWritingForDeleting error: &deleteError byAccessor: ^(NSURL *urlForModifying) {
                        //Create a remove error.
                        NSError *removeError;

                        //Delete the file.
                        [[NSFileManager defaultManager] removeItemAtURL: urlForModifying error: &removeError];

                        //If there was an error. Output it.
                        if (removeError) {
                            NSLog(@"%@", [removeError localizedDescription]);
                        }
                    }
                     ];

                    //If there was an error. Output it.
                    if (deleteError) {
                        NSLog(@"%@", [deleteError localizedDescription]);
                    }
                }
            }
        } else {
            // Replace this implementation with code to handle the error appropriately.
            // abort() causes the application to generate a crash log and terminate. You should not use this function in a shipping application, although it may be useful during development.

            /*
             Typical reasons for an error here include:
             * The parent directory does not exist, cannot be created, or disallows writing.
             * The persistent store is not accessible, due to permissions or data protection when the device is locked.
             * The device is out of space.
             * The store could not be migrated to the current model version.
             Check the error message to determine what the actual problem was.
             */
            NSLog(@"Unresolved error %@, %@", error, error.userInfo);
            abort();
        }
    }];

    //Return the container.
    return _persistentContainer;
}

Peritoneum

Answer 2

solidsnake4444's answer saves my day. Here is the Swift 5.0 version.

lazy var persistentContainer: NSPersistentContainer = {
    let container = NSPersistentContainer(name: "MyApp")
    let storeURL = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: "group.com.my.app")!.appendingPathComponent("MyApp.sqlite")

    var defaultURL: URL?
    if let storeDescription = container.persistentStoreDescriptions.first, let url = storeDescription.url {
        defaultURL = FileManager.default.fileExists(atPath: url.path) ? url : nil
    }

    if defaultURL == nil {
        container.persistentStoreDescriptions = [NSPersistentStoreDescription(url: storeURL)]
    }
    container.loadPersistentStores(completionHandler: { [unowned container] (storeDescription, error) in
        if let error = error as NSError? {
            fatalError("Unresolved error \(error), \(error.userInfo)")
        }

        if let url = defaultURL, url.absoluteString != storeURL.absoluteString {
            let coordinator = container.persistentStoreCoordinator
            if let oldStore = coordinator.persistentStore(for: url) {
                do {
                    try coordinator.migratePersistentStore(oldStore, to: storeURL, options: nil, withType: NSSQLiteStoreType)
                } catch {
                    print(error.localizedDescription)
                }

                // delete old store
                let fileCoordinator = NSFileCoordinator(filePresenter: nil)
                fileCoordinator.coordinate(writingItemAt: url, options: .forDeleting, error: nil, byAccessor: { url in
                    do {
                        try FileManager.default.removeItem(at: url)
                    } catch {
                        print(error.localizedDescription)
                    }
                })
            }
        }
    })
    return container
}()

Coir

This is a great solution for NSPersistentContainer. My app switches between NSPersistentContainer and NSPersistentCloudKitContainer depending if the user has iCloud toggled on. Unfortunately, when I'm using NSPersistentCloudKitContainer, I am unable to sync between my devices anymore. Any idea why this is happening ? – Danaus

Try this to fix the iCloud syncing issue https://mcmap.net/q/507489/-migrating-data-to-app-groups-disables-icloud-syncing – Mayhap

It works! But the folder .MyApp_SUPPORT/_EXTERNAL_DATA left. – Infirm

For me it seems like the old store is not deleted with this solution. The migration and deletion part gets called on every call. – Hydrochloride

I did so, but In extension I get nil here FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: "group.com.my.app") in real device? Do you know why? – Dowery

@Danaus try setting persistentStoreDescriptions.first.url only instead of the whole NSPersistentStoreDescription object. – Caldarium

May I ask why FileManager.default.fileExists(atPath: url.path) always returns false, but when I print out the url inside the code block, it shows the original store URL of the application? – Amply

If I remember correctly, the url tells you the path of the "default" path, but it doesn't means there is an existing store at that path. If your app is newly installed with the above code, it creates another store for shared storage purposes rather than using the default store. In that case, it makes much sense that FileManager.default.fileExists(atPath: url.path) always return false. – Coir

Answer 3

When moving NSPersistentCloudKitContainer, this is what worked for me. A couple things to watch out for:

• You need to ensure cloudKitContainerOptions is set on the persistent store description otherwise sync will stop working
• Do not call migratePersistentStore otherwise you'll end up with duplicate records, instead use replacePersistentStore - see this thread for more info
• You'll probably want to delete your old database once successfully migrated, destroyPersistentStore sounds promising but doesn't seem to actually delete the files - see this question

struct PersistenceController {
    static let shared = PersistenceController()

    let container: NSPersistentCloudKitContainer

    init(inMemory: Bool = false) {
        container = NSPersistentCloudKitContainer(name: "MyCuteDB")
        
        if inMemory {
            container.persistentStoreDescriptions.first!.url = URL(fileURLWithPath: "/dev/null")
        } else {
            // Use App Groups so app and extensions can access database
            let sharedStoreURL = FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: "group.com.yourdomain.yourapp")!.appendingPathComponent("\(container.name).sqlite")
            let defaultStoreURL = container.persistentStoreDescriptions.first!.url! //the URL always starts out in the default location

            // move database to shared location if needed
            if FileManager.default.fileExists(atPath: defaultStoreURL.path) && !FileManager.default.fileExists(atPath: sharedStoreURL.path) {
                let coordinator = container.persistentStoreCoordinator
                do {
                    try coordinator.replacePersistentStore(at: sharedStoreURL, destinationOptions: nil, withPersistentStoreFrom: defaultStoreURL, sourceOptions: nil, ofType: NSSQLiteStoreType)
                    try? coordinator.destroyPersistentStore(at: defaultStoreURL, ofType: NSSQLiteStoreType, options: nil)
                    
                    // destroyPersistentStore says it deletes the old store but it actually truncates so we'll manually delete the files
                    NSFileCoordinator(filePresenter: nil).coordinate(writingItemAt: defaultStoreURL.deletingLastPathComponent(), options: .forDeleting, error: nil, byAccessor: { url in
                        try? FileManager.default.removeItem(at: defaultStoreURL)
                        try? FileManager.default.removeItem(at: defaultStoreURL.deletingLastPathComponent().appendingPathComponent("\(container.name).sqlite-shm"))
                        try? FileManager.default.removeItem(at: defaultStoreURL.deletingLastPathComponent().appendingPathComponent("\(container.name).sqlite-wal"))
                        try? FileManager.default.removeItem(at: defaultStoreURL.deletingLastPathComponent().appendingPathComponent("ckAssetFiles"))
                    })
                } catch {
                    //TODO: Handle error
                }
            }

            // change URL from default to shared location
            container.persistentStoreDescriptions.first!.url = sharedStoreURL

            let description = container.persistentStoreDescriptions.first!
            description.cloudKitContainerOptions = NSPersistentCloudKitContainerOptions(containerIdentifier: "iCloud.com.yourdomain.yourapp")
        }
        
        container.loadPersistentStores { description, error in
            //TODO: Handle error
        }
        container.viewContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy
        container.viewContext.automaticallyMergesChangesFromParent = true
    }
}

Goulet

Any open source packages to recommend that can perform such migration? – Radmen

Answer 4

UPDATE:

To migrating an existing persistent store, the NSPersistentContainer contains the persistentStoreCoordinator, an instance of NSPersistentStoreCoordinator. This exposes the method migratePersistentStore:toURL:options:withType:error: to migrate a persistent store.

I would do the following:

// Get the reference to the persistent store coordinator
let coordinator = persistentContainer.persistentStoreCoordinator
// Get the URL of the persistent store
let oldURL = persistentContainer.persistentStoreDescriptions.url
// Get the URL of the new App Group location
let newURL = NSFileManager.defaultManager().containerURLForSecurityApplicationGroupIdentifier("YOUR_APP_GROUP")
// Get the reference to the current persistent store
let oldStore = coordinator.persistentStore(for: oldURL)
// Migrate the persistent store
do {
   try coordinator.migratePersistentStore(oldStore, to: newURL, options: nil, withType: NSSQLiteStoreType)
} catch {
   // ERROR
}

Please note that the above hasn't been tested and I haven't handled Optionals so it isn't complete. Also, I apologise for it being in Swift. Hopefully it is easy enough for you to write the equivalent in Objective-C.

ORIGINAL:

The following outlines how to create an NSPersistentContainer interfacing to a persistent store in a non-default location.

The NSPersistentContainer exposes the defaultDirectoryURL, and states:

This method returns a platform-dependent NSURL at which the persistent store(s) will be located or are currently located. This method can be overridden in a subclass of NSPersistentContainer.

If you subclass NSPersistentContainer and define the defaultDirectoryURL to be an App Group directory using containerURLForSecurityApplicationGroupIdentifier, you should then be able to access the container between your application and extensions (assuming they have the same App Group entitlements).

NSPersistentContainer also exposes persistentStoreDescriptions which also features a URL instance. Likewise, you may be able to update this to the App Group URL before calling loadPersistentStoresWithCompletionHandler:.

Note that I have not used NSPersistentContainer, and have no idea whether this sharing would cause any concurrency issues.

Everett

I'm not sure I understand what you are saying. At what point is the file moved from the normal location to the shared location? – Peritoneum

Sorry about that. I have updated my answer to explain how to actually migrate the persistent store. Hopefully that helps! – Everett

Sorry, I haven't had a chance to get to my mac yet to try it. I'm hoping to tomorrow. – Peritoneum

I'm still confused on where to put your new edited code compared to the functions I already have. Would I do it in the completion handler of the loadPersistentStoresWithCompletionHandler function? If so then the second time around, how do I know NOT to do the migration again and how would I know load is now looking inside the new app group? – Peritoneum