Lightweight migration of a NSPersistentDocument

Question

I'm trying to do a lightweight migration of a SQLite store in Core Data. Working on Lion 10.7.3 with Xcode 4.3.1.

In my NSPersistentDocument subclass (AccountDocument), I've overridden the method used to configure the persistent store coordinator so that it gets the proper options for the migration:

- (BOOL)configurePersistentStoreCoordinatorForURL:(NSURL *)url ofType:(NSString *)fileType modelConfiguration:(NSString *)configuration storeOptions:(NSDictionary *)storeOptions error:(NSError **)error
{
    NSMutableDictionary *newStoreOptions;
    if (storeOptions == nil) {
        newStoreOptions = [NSMutableDictionary dictionary];
    }
    else {
        newStoreOptions = [storeOptions mutableCopy];
    }
    [newStoreOptions setObject:[NSNumber numberWithBool:YES] forKey:NSMigratePersistentStoresAutomaticallyOption];
    [newStoreOptions setObject:[NSNumber numberWithBool:YES] forKey:NSInferMappingModelAutomaticallyOption];

    BOOL result = [super configurePersistentStoreCoordinatorForURL:url ofType:fileType modelConfiguration:configuration storeOptions:newStoreOptions error:error];
    return result;
}

(Thanks to Malcolm Crawford for that tip: http://homepage.mac.com/mmalc/CocoaExamples/controllers.html)

When I run the app, it fails in NSPersistentDocument's implementation of -managedObjectModel:

* thread #1: tid = 0x2703, 0x00007fff931d9350 libobjc.A.dylib`objc_msgSend_vtable13 + 16, stop reason = EXC_BAD_ACCESS (code=13, address=0x0)
    frame #0: 0x00007fff931d9350 libobjc.A.dylib`objc_msgSend_vtable13 + 16
    frame #1: 0x00007fff8935e975 CoreData`-[NSKnownKeysDictionary1 _setValues:retain:] + 197
    frame #2: 0x00007fff8935f288 CoreData`_newReadModelFromBytes + 648
    frame #3: 0x00007fff8935b93e CoreData`+[NSManagedObjectModel(_NSManagedObjectModelPrivateMethods) _newModelFromOptimizedEncoding:error:] + 9310
    frame #4: 0x00007fff89359451 CoreData`-[NSManagedObjectModel(_NSManagedObjectModelPrivateMethods) initWithContentsOfOptimizedURL:] + 305
    frame #5: 0x00007fff89358d7b CoreData`-[NSManagedObjectModel initWithContentsOfURL:] + 443
    frame #6: 0x00007fff893e9519 CoreData`+[NSManagedObjectModel mergedModelFromBundles:] + 377
    frame #7: 0x00007fff8ded7037 AppKit`-[NSPersistentDocument managedObjectModel] + 301
    frame #8: 0x00007fff8ded70b3 AppKit`-[NSPersistentDocument managedObjectContext] + 75
    frame #9: 0x00007fff8ded6e3f AppKit`-[NSPersistentDocument _persistentStoreCoordinator] + 18
    frame #10: 0x00007fff8ded6b5d AppKit`-[NSPersistentDocument configurePersistentStoreCoordinatorForURL:ofType:modelConfiguration:storeOptions:error:] + 51
    frame #11: 0x0000000100003193 BeanCounter`-[AccountDocument configurePersistentStoreCoordinatorForURL:ofType:modelConfiguration:storeOptions:error:] + 419 at AccountDocument.m:298

From what I can tell from the documentation, the default implementation looks something like this:

- (id)managedObjectModel
{
    NSManagedObjectModel *result = [NSManagedObjectModel mergedModelFromBundles:nil];
    return result;
}

So to debug the problem a little more, I overrode that method with this:

- (id)managedObjectModel
{
    NSBundle *bundle = [NSBundle mainBundle];
    NSURL *url = [bundle URLForResource:@"AccountDocument2" withExtension:@"momd"];
    NSManagedObjectModel *result = [[[NSManagedObjectModel alloc] initWithContentsOfURL:url] autorelease];  
    return result;
}

(Thanks to Jeff LaMarche for the idea: http://iphonedevelopment.blogspot.com/2009/09/core-data-migration-problems.html)

The bundle and url both point to the places I expect (and I've followed Marcus Zarra's advice to clean the project so there aren't any stray .mom or .momd bundles in the application package: Using mergedModelFromBundles: and versioning (CoreData)). Yet the app continues to crash while loading the model from the url.

I've checked that the AccountDocument2.xcdatamodeld is a package that has two models for versioning: AccountDocument 2.xcdatamodel and (the original) AccountDocument.xcdatamodel. The "Versioned Core Data Model" popup menu in the file properties is set to "AccountDocument 2".

The only difference between the two models is that one Entity has an additional (and optional) Attribute. My understanding is that qualifies the model for a lightweight migration.

Obviously, I'm doing something wrong here, but I have no idea what. Any help would be most appreciated…

Update:

Per Martin's suggestion (and a check of the NSPersistentDocument documentation) I tried using this code for the accessor:

- (id)managedObjectModel
{
    static id sharedManagedObjectModel = nil;

    if (sharedManagedObjectModel == nil) {
        NSBundle *bundle = [NSBundle mainBundle];
        NSURL *url = [bundle URLForResource:@"AccountDocument2" withExtension:@"momd"];
        sharedManagedObjectModel = [[NSManagedObjectModel alloc] initWithContentsOfURL:url];
    }

    return sharedManagedObjectModel;
}

Still crashing…

Update

After some suggestions on Twitter, I upgraded to Xcode 4.3.2, but the problems persist.

RAGE UPDATE

I just created the versioned model package (AccountDocument2.xcdatamodeld) using Xcode 4.2 on Snow Leopard. After building and running the app, everything works as expected.

I then took the AccountDocument2.xcdatamodeld file package back over to Lion and Xcode 4.3.2. When I build and run the app, it continues to crash while loading the .momd resource. Yes kids, that means Xcode 4.3.x and the Data Model Compiler (MOMC) are to blame. I don't see a workaround other than doing all my builds on Snow Leopard.

I'm not one to bash Xcode 4, but when we find ourselves in a situation where the toolchain can't produce an opaque file (.mom and .momd) from an opaque specification (.xcdatamodel and .xcdatamodeld) it's pretty hard to be upbeat about the state of Mac and iOS tools. It's ridiculous that a core component of these platforms breaks to the degree where I can't build and run my app on the latest version of the SDK and developer tools.

It's Come To This Update

More proof that this is a serious bug with the Data Model Compiler (MOMC) in Xcode 4.3.2: if I copy the .momd bundle from the Resource folder created by Xcode 4.2 into my project and add them to the build as a Copy Files build phase, the application works fine.

I also did some tests where I removed validation rules and default values for the Attributes of the various Entities (based on Marcus' suggestion below.) No change, the compiler still creates an invalid .momd. I also tried creating a versioned model where NOTHING was changed: the compiled .momd continued to crash. So whatever you have in your current models (and data they represent) is the source of the problem.

Also of note: this bug is not isolated to NSPersistentDocument (as I originally thought when I started this question.) I can cause an app to crash by just using [[NSManagedObjectModel alloc] initWithContentsOfURL:modelURL].

For now, I'll be editing/versioning my models using Xcode 4.2 on Snow Leopard and moving the compiled resources over to Xcode 4.3.2 on Lion. If you use Core Data in any way, I suggest you do the same until this bug is addressed. Trust me, you'll spend days trying to figure out what the hell is going on if you don't.

Now to submit a Radar…

Radar Update

I just submitted this Radar:

http://www.openradar.me/11184500

The Oh Crap It Must Be Lion Update

I just downloaded and installed the Xcode 4.2 for Lion tools from http://developer.apple.com/downloads. The sample application used in the Radar still crashes.

(Note: you can't install Xcode 4.2.1 because the certificate used to sign the DeveloperTools.pkg has expired. Only Xcode 4.2 will work.)

If you're under NDA you'll also find that the beta tools aren't helpful either.

Hope you have a copy of Snow Leopard with Xcode 4.2 sitting around: http://furbo.org/2012/03/28/vmware-for-developers/

The WTF Do Fetch Requests Have To Do With Versioned Entities and Attributes Update

Via Evadne Wu on Twitter:

https://twitter.com/#!/evadne/status/187625192342818818

And how she did it:

https://twitter.com/#!/evadne/status/187629091518816258

(.mom files are binary plists.)

The root of the problem is a single Fetch Request. How that figures into a migration of data from one model to another is for an engineer at Apple to figure out.

Remediless

Are you using ARC in the class that defines configurePersistentStoreCoordinatorForURL:::::? If not, you're leaking the copy you make of storeOptions. – Demarcate

Depending on whether the existing storeOptions is present, newStoreOptions will be either retained (if you call [storeOptions mutableCopy]) or autoreleased if you follow the path that calls [NSMutableDictionary dictionary] – Geralyngeraniaceous

Peter's right that there's a leak of newStoreOptions, but it doesn't really matter since I just checked the code path and storeOptions was coming in as nil. This isn't the source of the problem while loading the model anyway (the models can't be loaded, much less migrated.) – Remediless

@Remediless You probably want to submit your last update or two as an answer instead of as an edit to the original post. – Achondrite

We ran into this issue or an incredibly similar one, but found an easier way to work around it, but equally enfuriating. When lightweight migration failed for us, we added a mapping model to tinker around. Fine. Then we deleted it to go back to trying lightweight migration and sure enough it worked. After going back and forth and trying it over and over again (from a fresh start) we could reliably prove just the act of adding the mapping model changed the model file somehow to where lightweight would work. Grrrrr. – Tetragonal

@StephenTallent Thanks for that tip, but unfortunately it doesn't work in this case. Still stuck… – Remediless

I think it would be useful to write an actual answer with a full explanation rather than simply linking to those tweets. Then, mark the answer as such for future people who come to this question. – Pester

Keep going Craig, I think at some point you're going to find @interface NSManagedObject (NSManagedObjectFetchBugApology) - (void)heyCraig:(id)in1 sorryAboutThat:(id)in2 mustHaveSlippedTheUnitTests:(id)in3 theRootCauseIsBothVerySimpleAndVeryComplicated:(id)in4 thanksForTheDetailedRadarWeOweYouABeerAtNextWWDCSeeYouThere:(id)in5; @end – Isaacson

Answer 1

The compiled .momd resources can be loaded after an "existingPartner" fetch request is changed from:

name == $name

to:

name == $NAME

It's counterintuitive that a part of the object model that does not affect the persistence of data breaks versioning and lightweight migration. It's clear from the documentation that this should not be the case:

Core Data’s perspective on versioning is that it is only interested in features of the model that affect persistence.

Use the power of the CHOCKLOCK to fix your Fetch Requests or delete them completely and rely on NSPredicates created in code.

Remediless

Answer 2

I think you need to store the managed object model in an instance variable. You are returning an autoreleased object, which is what is probably causing the bad access.

Kagoshima

Updated my code examples above and unfortunately this doesn't help. – Remediless

Answer 3

Based on your theory that it is an issue with the MOMC, Do you have any validation rules in the mom?

I have seen reports where validation rules do not survive the 4.x MOMC.

Patrolman

Thanks for the tip, Marcus. I tried removing the validation rules from both original and versioned models and it still crashes on load. – Remediless

Answer 4

This might be somewhat related to an issue I had with using Fetch Requests when iOS5 first came out in beta. It was causing a build warning and would crash the app on launch. I wasn't really using the fetch request so I removed it and everything worked fine: Core Data warning: "Version hash information not available for all models"

Aila