How to debug/handle intermittent "authorization denied" and "disk i/o" errors when adding SQL store to an NSPersistentStoreCoordinator?

Question

I have an app in the app store and am using a logging service to get crash logs and associated log data. I am seeing an intermittent crash (low # of users affected and low # of crashes per user) but it is baffling me.

What happens in these crashes is the following:

1. App launches and initializes Core Data stack

2. App attempts to add a SQL store to the NSPersistentStoreCoordinator with the following code (storeURL is valid):

   NSDictionary *options = @{
       NSMigratePersistentStoresAutomaticallyOption : @(YES),
       NSInferMappingModelAutomaticallyOption : @(YES)
   };
   
   sqlStore = [_persistentStoreCoordinator addPersistentStoreWithType:NSSQLiteStoreType
                                                        configuration:nil
                                                                  URL:storeURL
                                                              options:options
                                                            error:&error];
   

3. One of the following errors occur when adding this store:

NSError:

Domain=NSCocoaErrorDomain
Code=256 "The operation couldn’t be completed. (Cocoa error 256.)"
UserInfo=0x1dd946a0 {NSUnderlyingException=authorization denied, NSSQLiteErrorDomain=23}

or

NSError:

Domain=NSCocoaErrorDomain
Code=256 "The operation couldn’t be completed. (Cocoa error 256.)"
UserInfo=0xc6525d0 {NSUnderlyingException=disk I/O error, NSSQLiteErrorDomain=10}

After this condition, the app will crash b/c the SQL store is required for the app to function. I could attempt to gracefully handle this failure by trying a new storeURL but I don't want the user to lose existing data. Also, I have never personally reproduced this issue and based on the low number of users affected and crash logs I believe it is a low impact problem and does not recur on a subsequent app launch.

I'm hoping there's a Core Data guru out there with some suggestions on how to debug and prevent/handle these conditions. My core data stack initialization code is straight from the xcode project generator and I have ruled out any concurrency issues in that the persistent store coordinator is only initialized once (on launch) and this error occurs in this initialization.

Happy to provide more code/info if relevant.

thanks!

Oho

Is some of your code removing / deleting the database file(s)? Smells like the file is open while you're deleting it. If you're deleting, you also need to make sure to delete all files, not just the main file. Otherwise the DB will look corrupted. – Oryx

There is no code that directly does anything with the database file other than in the code I included. Once the DB is open all data changes are done through managed object context(s). Also, while this does seem like a corruption of some kind it only appears to happen once and doesn't recur on subsequent launches. As I haven't repro'ed myself this is conjecture based on analyzing crash logs. – Oho

Are you using any kind of encryption on the DB (i.e. NSPersistentStoreFileProtectionKey)? – Oryx

No encryption. My posted code shows all the options used when adding the store. – Oho

Answer 1

It looks like XJones and I have been able to locate the cause of this. It appears to be an iOS edge-case bug or undocumented behavior. I've filed this under Apple bug ID 12935031.

There is an unaccounted for scenario where an application using Core Location significant location change or region monitoring can fail to launch properly (or have other unintended consequences) due to the fact that as of iOS 5, Core Data stores use data protection (encryption) by default.

Steps to Reproduce:

1) Turn on pass code protection on the device

2) Create an application that starts Significant Location Monitoring or Region Monitoring and keeps it started even when in the background. Ie. An app that uses background significant location change or region monitoring.

3) Wait for the battery on the device to run out (there may be other causes as well)

4) The device will shut down

5) Connect the device to the Mac

6) Once the charge is adequate, the device will boot. Important: Do not unlock device at this time.

7) Exit or enter the monitored range or cause a significant change in location to occur. The device will now automatically relaunch the application, because it registered for Significant Location Monitoring or Region Monitoring

8) However, since the device has not been unlocked by the user (pass code not yet entered), the application will not be able to read any of its protected data files. In a Core Data application, this will cause the persistent store coordinator to fail to add the persistent store file to the managed object context. This will cause the app to crash or depending on the code used by the developer even attempt to reset the database. In other apps it may cause crashes for other reasons as this is an unexpected, undocumented side effect of the data protection feature being turned on by default for Core Data stores in iOS 5 and later.

The solution or workaround until Apple corrects this or at least documents it is to either make sure your application stops monitoring significant location changes/regions in applicationWillTerminate or to turn off the default data protection feature by setting NSFileProtectionNone for the NSFileProtectionKey key in the options dictionary when adding the Core Data store to the persistent store coordinator. to wait for the file store to become available using a while() loop that checks for the protected data to become available. There may be other ways of doing this using KVO, but this method works reliably and is easiest to insert into existing code without reworking your entire application startup process.

Update: looks like just setting that key is not enough if data protection is already active on the store. You have to manually set it:

[[NSFileManager defaultManager] setAttributes:[NSDictionary dictionaryWithObject:NSFileProtectionNone forKey:NSFileProtectionKey] ofItemAtPath:storePath error:nil];

Here's the fix for apps that need background location monitoring, thanks to more input from XJones and a bit more research in Apple's scattered docs:

while(![[UIApplication sharedApplication] isProtectedDataAvailable]) {
    [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.5f]];
}

This code goes in before you attempt to add the store data file to the persistent store coordinator.

Update 2015: You can also set NSPersistentStoreFileProtectionKey to NSFileProtectionNone. This will properly disable file protection (if you don't need it) and just work without requiring any workarounds. The reason it did not work in previous attempts is because the dictionary key I was testing with was incorrect.

Galan

thanks for all your help. In my case, I am using the significant change location service and intend for it to be disabled when the app is terminated or in the background but I had a bug where there are cases the the background location services remains active. – Oho

Bad news, the reproduction process is correct but the fix seems to be ineffective. – Galan

Can you elaborate? Do you mean setting the file protection to none isn't effective after the fact? In my case, I don't want my app launching into the background anyway so if I can fully prevent that then the problem would be eliminated. – Oho

It seems setting file protection to none has no effect, even on a new file. Even tried setting it to none on quit using NSFileManager and still no change. Your fix of shutting down location monitoring should still work. But in my case, the problem cannot be solved this way as I specifically need the background monitoring. – Galan

see the new answer I just added. UIApplication provides notifications relevant to accessing protected data. – Oho

stackoverflow's formatting is really messed up for some reason lately, getting code formatted right is a nightmare :-/ – Galan

sorry guys but is it the case that with this solution the app polls until user unlocks the device? this can take a while if I understand it correctly. – Mclean

@arashpa - unfortunately that is the only way right now, since otherwise the data file is unreadable. It might make sense to increase the polling interval to save battery life though (from the 0.5f above). Its still a bug as when we specify we don't want data protection on the Core Data store, it is still used, causing this issue. I've already filed a bug report with Apple but so far no response has been received. – Galan

+1 This also seems to be the problem with VoIP apps at startup. – Kippie

Answer 2

I just noticed the following notifications in the UIApplication docs:

UIApplicationProtectedDataDidBecomeAvailable UIApplicationProtectedDataWillBecomeUnavailable

These are available in iOS 4.0 and later. @lupinglade, I think you need to observe these notifications and only access your protected files (i.e. the store) after you recieve UIApplicationProtectedDataDidBecomeAvailable.

Oho

Here's the fix, this goes in the code before attempting to add the store file to the persistentstorecoordinator: while(![[UIApplication sharedApplication] isProtectedDataAvailable]) { [[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.5f]]; } – Galan

Answer 3

The solution @lupinglade pretty much worked in my case (VoIP app had when starting up from a reboot), but I would also like to point out that there's a delegate function for AppDelegate that is called when protected data becomes available:

applicationProtectedDataDidBecomeAvailable:

This made it a bit easier to implement the solution in my case.

According to the header file it's available from iOS 4 onwards.

Kippie

Answer 4

Same problem here, still have not been able to find a solution. I've found it seems to be related to having a lock code set on the device. WIthout the lock code I could never reproduce the error, now I have been able to a bunch of times. Console log is:

Error is: Error Domain=NSCocoaErrorDomain Code=256 "The operation couldn’t be completed. (Cocoa error 256.)" UserInfo=0x1fd80110 {NSUnderlyingException=authorization denied, NSSQLiteErrorDomain=23}

File does exist and not using any encryption.

Galan

What are steps you use to reproduce this problem in your app? – Oho

Does your app use location services or some other background app type? It seems that this combination of factors is needed to trigger this issue and reproduce it: - Passcode protection must be enabled in iOS Settings - Your app must use location monitoring, in my case its region monitoring - Your app must be launched at least once to start the location change monitoring - The iPhone must run out of battery and shut down on its own - Connect the iPhone to the Mac to charge - When the iPhone boots back up once the battery has a bit of charge, this error will occur when iOS relaunches your app – Galan

It seems that when iOS relaunches your app for the significant location or region monitoring changes, it launches it too early or in some wrong "mode", causing the persistentstorecoordinator to fail to add the store. I am still testing, as its taking some time to debug, due to the complexity of the reproduction process. Simply force quitting (or if it crashes)/relaunching the app after this, makes everything work again. Provided your code does not move away or delete the sqlite file when the error occurs. Perhaps it has to do with encryption when using the pass code lock feature. – Galan

I have been working on this bug for about a year now, only lately have I been getting close and able to reproduce it. – Galan

More details: - Since I don't use the pass code lock feature, I could not ever reproduce this issue until I decided to turn it on because some customers who had this issue said they were using it. - The issue was finally reproduced when my test phone's battery ran out and I connected it to charge - I have been able to reproduce it 4 or 5 times now this exact way - Customers that have this issue have a location (region) monitoring option enabled in our app - The error is almost certainly associated with iOS automatically restarting the app on boot after battery is fully discharged – Galan