UIDocument & NSFileWrapper - large files taking a long time to save, despite incremental changes

Question

I have a UIDocument based app that uses NSFileWrappers to store data. The 'master' file wrapper contains many additional directory file wrappers, each of which represents a different page of the document.

When saving a large document for which only a small proportion of one page has been modified, UIDocument spends a LONG time in the background writing the changes (in writeContents:andAttributes:safelyToURL:forSaveOperation:error:). Surely it should only be writing out this one small change to the file wrapper... what's taking so long?

My contentsForType:error: override returns a new directory file wrapper with the contents of the master file wrapper (à la WWDC 2012 Session 218 - Using iCloud with UIDocument):

- (id)contentsForType:(NSString *)typeName error:(NSError *__autoreleasing *)outError
{
    if (!_fileWrapper) {
        [self setupEmptyDocument];
    }
    return [[NSFileWrapper alloc] initDirectoryWithFileWrappers:[_fileWrapper fileWrappers]];
}

And here's a lovely picture of a stack trace from Time Profiler:

Incidentally, it says ~1.6s in that worker thread to save - in actual run time this equated to about 8 seconds.

---

Edit:

Is there some way I can check whether the file wrappers require writing to disk or not? Just so I can confirm that I'm not somehow doing something strange like updating every sub file wrapper when I make a small change (although I'm sure I'm not...).

---

Edit:

I had a further play around with the CloudNotes sample app, and it appears that NSFileWrapper does implement incremental saving, at least in that case! I tested it by initialising a document with 100 notes, each of which contained about 5MB of data. I did a small edit here and there (a single character change to a text view flags the document as needing saving), and recorded roughly how long each save took. The test is relatively crude (and run on the simulator), but the results were something like this:

• 1st write: ~8000ms
• 2nd write: ~4000ms
• 3rd write: ~300ms
• all subsequent writes: ~40ms

Obviously there are many factors affecting the time it takes, especially since it's saving using file coordination in a background thread, but in general the trend always seems to be this sort of exponential decay, until all writes become really very fast.

But I'm still trying to figure out why this doesn't happen in my app. For a large multi-page document (large, but still many times smaller than the document for the CloudNotes test I performed above) the user can be waiting many seconds for a document to close. I don't want to have to put a spinner up for something that should be practically instantaneous.

Merger

Are you doing any auto-saving, or are you saving everything at once? Are you initiating the save from the main thread? Are you replacing every NSFileWrapper, or only the ones that have actually changed? – Weeds

I'm relying solely on autosaving, so the saves are initiated by UIDocument, doing the usual thing of calling contentsForType:error: (main thread) before writing (background thread). I am only replacing the file wrappers I have changed (when they change). However, I have noticed I'm using a method that looks at all my page file wrappers and sorts them into an ordered array. I think this is messing with NSFileWrapper's lazy loading and causing them to need writing somehow. I'm just implementing an index so that I don't have to do this, and will see what difference it makes. – Merger

Nope, changing to just using an index file to keep track of pages/page numbers has not helped. Still trying to track down what could be causing the slow saving. – Merger

You can join me in the chat here. – Weeds

Did you ever find out exactly what you were missing? – Cranky

@Cranky Unfortunately not. – Merger

Answer 1

NSFileWrapper is actually loading the entire document into memory. So with a UIDocument, using an NSFileWrapper is actually not good for large documents. The documentation makes you think it does incremental saving, but in my case it didn't seem to do that.

UIDocument isn't restricted to just NSFileWrapper or NSData. You can use your own custom class, you just have to override certain methods. I ended up writing my own file wrapper class that simply refers to files on disk and reads/writes individual files on-demand.

This is what my UIDocument class looks like using the custom file wrapper:

@implementation LSDocument

- (BOOL)writeContents:(LSFileWrapper *)contents
        andAttributes:(NSDictionary *)additionalFileAttributes
          safelyToURL:(NSURL *)url
     forSaveOperation:(UIDocumentSaveOperation)saveOperation
                error:(NSError *__autoreleasing *)outError
{
    return [contents writeUpdatesToURL:self.fileURL error:outError];
}

- (BOOL)readFromURL:(NSURL *)url error:(NSError *__autoreleasing *)outError
{
    __block LSFileWrapper *wrapper = [[LSFileWrapper alloc] initWithURL:url isDirectory:NO];
    __block BOOL result;
    dispatch_sync(dispatch_get_main_queue(), ^(void) {
        result = [self loadFromContents:wrapper
                                 ofType:self.fileType
                                  error:outError];
    });
    [wrapper loadCache];
    return result;
}

@end

I use this as a base class and subclass it for other projects. It should give you an idea of what you have to do to integrate a custom file wrapper class.

Weeds

Thanks for taking the time to answer (both of my related questions!). I've spent so much time trying to look into the details of NSFileWrapper, but there is so little information about them. A number of things confuse me: 1) If file wrappers load all data into memory, and don't support incremental saving, what is the point of them? 2) The documentation doesn't just "make you think" they do incremental saving, it says it outright! From the "Document-Based App Programming Guide for iOS": "File wrappers support incremental saving. In contrast with a single binary data object, if a file ... – Merger

...wrapper contains your document data - for example, text and an image - and the text changes, only the file containing the text has to be written out to disk. This capability results in better performance." – Merger

File NSFileWrapper's are immutable, so I think it may save file wrappers that have been replaced. So if you replace an NSFileWrapper even though it changed it may save it anyway. An NSFileWrapper loads everything into memory, so in my case that wasn't desirable - so I made my own that refers to files instead w/ a BOOL to indicate I wanted to cache certain files in memory. When I update a file it sets a "content" ivar with an "updated" BOOL. When my parent is saved it looks for "updated" and saves the updated files and sets "content" to nil if the "cache" BOOL is false. – Weeds

Your implementation sounds good, but I only want to stray beyond NSFileWrapper as a last resort. My understanding was that custom incremental read/write solutions were mainly for the case that the document contains data blobs (e.g. video files). My documents consist of many very small files, and even when the total document size is > 100MB I don't see any kind of performance issues when reading - just when writing. This suggests some kind of NSFileWrapper optimisation to me, but I can't back that up with research. Have you seen evidence that NSFileWrapper loads everything into memory? – Merger

I believe there are a few options, like memory-mapping, but ultimately that's how it works. The documentation is not entirely clear. I've open sourced my file wrapper here: github.com/lukescott/LSFileWrapper – Weeds

@Merger I'm wondering what have you chosen eventually? I'm in seven years later facing exactly the same problem... – Sayed

Answer 2

I know this is a super old thread, but to help future travelers: In my case, I had a subdirectory NSFileWrapper which was not incrementally saving.

I found that if you make a copy of an NSFileWrapper, you need to set the copy's filename, fileAttributes (and possibly preferredFilename) to the original's in order for the save to be incremental. After copying those over, the contents of the subfolder would incrementally save (i.e. only write if replaced with new NSFileWrappers).

Note to Apple: Seriously, the whole NSFileWrapper API is a mess and should be cleaned up.

Janayjanaya

There is a super brief mention of "using flat struct in package" in (maybe) "bundle design guideline". Could putting images in a directory, instead of 'flat' putting them directly in the package, is one of the cause for not incrementally saving? – Sayed