I need to upload lot of files (up to 100) to server using AFNetworking in a single streaming request that can provide me upload progress

Question

There are couple of challenges I am facing with streaming request using AFNetworking 2.0.

I want to upload lot of files (~50MB) to server and the request HAVE TO BE STREAMING. (otherwise app will crash due to memory pressure)

I have tried various ways to do it in AFNetworking 2.0 without any success. This is what I am doing currently:

NSMutableURLRequest *request = [[AFHTTPRequestSerializer serializer] multipartFormRequestWithMethod:@"POST" URLString:baseServiceUrl parameters:nil constructingBodyWithBlock:^(id<AFMultipartFormData> formData) {
    int imageIndex = 0;
    for (NSURL *tempFileUrl in tempFilesUrlList) {
        NSString* fileName = [NSString stringWithFormat:@"image%d", imageIndex];
        NSError *appendError = nil;
        [formData appendPartWithFileURL:tempFileUrl name:fileName error:&appendError];
        imageIndex++;
    }
} error:&error];

AFHTTPRequestOperation *requestOperation = nil;
requestOperation = [manager HTTPRequestOperationWithRequest:request success:^(AFHTTPRequestOperation *operation, id responseObject) {
    DLog(@"Uploading files completed: %@\n %@", operation, responseObject);
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
    DLog(@"Error: %@", error);
}];

[requestOperation setUploadProgressBlock:^(NSUInteger bytesWritten, long long totalBytesWritten, long long totalBytesExpectedToWrite) {
    double percentDone = (double)totalBytesWritten / (double)totalBytesExpectedToWrite;
    DLog(@"progress updated(percentDone) : %f", percentDone);
}];
[requestOperation start];

This works fine, but its not streaming. It prepares the request in memory can crashes if request is too big. I had experimented with native sockets and streams, but Apple says they may disapprove the app because of it.

Another approach I have tried is following:

AFHTTPSessionManager* manager = [AFHTTPSessionManager manager];
NSString *appID = [[NSBundle mainBundle] bundleIdentifier];
NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration backgroundSessionConfiguration:appID];
NSURL* serviceUrl = [NSURL URLWithString:baseServiceUrl];
manager = [manager initWithBaseURL:serviceUrl sessionConfiguration:configuration];


NSURLSessionDataTask *uploadFilesTask = [manager POST:baseServiceUrl parameters:nil constructingBodyWithBlock:^(id<AFMultipartFormData> formData) {
    int imageIndex = 0;
    for (NSURL *tempFileUrl in tempFilesUrlList) {
        NSString* fileName = [NSString stringWithFormat:@"image%d", imageIndex];
        NSError *appendError = nil;
        [formData appendPartWithFileURL:tempFileUrl name:fileName error:&appendError];
        imageIndex++;
    }
} success:^(NSURLSessionDataTask *task, id responseObject) {
    DLog(@"Files uploaded successfully: %@\n %@", task, responseObject);
} failure:^(NSURLSessionDataTask *task, NSError *error) {
    DLog(@"Error: %@", error);
}];

[progressBar setProgressWithUploadProgressOfTask:uploadFilesTask animated:true];

But this gives me run time error saying:

Terminating app due to uncaught exception 'NSGenericException', reason: 'Upload tasks in background sessions must be from a file' *** First throw call stack: ( 0 CoreFoundation 0x02cc75e4 __exceptionPreprocess + 180 1 libobjc.A.dylib
0x02a4a8b6 objc_exception_throw + 44 2 CFNetwork
0x0459eb6c -[__NSCFBackgroundSessionBridge uploadTaskForRequest:uploadFile:bodyData:completion:] + 994 3
CFNetwork 0x045f2f37 -[__NSCFURLSession uploadTaskWithStreamedRequest:] + 73

(Must be streaming, and able to continue in background)

Rasmussen

To my knowledge iOS doesn't support streaming with background uploads. Do you need to support this in the background or only foreground? If in the foreground you in theory should be able to stream? The other option is to look at server side support for uploading files in smaller parts. Does your server support multi-part uploads (i.e. Amazon S3's multi-part upload docs.aws.amazon.com/AmazonS3/latest/dev/mpuoverview.html ). Using parts you would be able to upload files in smaller parts which avoid the memory overhead. – Ianiana

Answer 1

I solved this myself recently.

I recommend using a networking library such as AFNetworking. It will save you from doing monotonous tasks that have been solved before.

In order to truly send the request in the background you'll need to write the entire request to a file, and then stream that to your server using NSURLSession. (That is the accepted answer -- link below) So, you'll need to write the request to disk without reading entire files into memory.

How to upload task in background using afnetworking

You can stream a file (not in background) like:

NSMutableURLRequest *request = [[AFHTTPRequestSerializer serializer] multipartFormRequestWithMethod:@"POST" URLString:@"http://example.com/upload" parameters:nil constructingBodyWithBlock:^(id<AFMultipartFormData> formData) {
        [formData appendPartWithFileURL:[NSURL fileURLWithPath:@"file://path/to/image.jpg"] name:@"file" fileName:@"filename.jpg" mimeType:@"image/jpeg" error:nil];
    } error:nil];

AFURLSessionManager *manager = [[AFURLSessionManager alloc] initWithSessionConfiguration:[NSURLSessionConfiguration defaultSessionConfiguration]];
NSProgress *progress = nil;

NSURLSessionUploadTask *uploadTask = [manager uploadTaskWithStreamedRequest:request progress:&progress completionHandler:^(NSURLResponse *response, id responseObject, NSError *error) {
    if (error) {
        NSLog(@"Error: %@", error);
    } else {
        NSLog(@"%@ %@", response, responseObject);
    }
enter code here
}];

[uploadTask resume];

http://cocoadocs.org/docsets/AFNetworking/2.5.0/

Note: A simple solution is to request time for your app to run in the background. When applicationDidEnterBackground: is called on your app delegate you can call [[UIApplication sharedApplication] beginBackgroundTaskWithExpirationHandler:] to ask for some time to run in the background. ([[UIApplication sharedApplication] endBackgroundTask:]; to say you're done.) What are the drawbacks you ask? You are given a limited amount of time. "If you do not call endBackgroundTask: for each task before time expires, the system kills the app. If you provide a block object in the handler parameter, the system calls your handler before time expires to give you a chance to end the task."

https://developer.apple.com/library/ios/documentation/UIKit/Reference/UIApplication_Class/#//apple_ref/occ/instm/UIApplication/beginBackgroundTaskWithExpirationHandler:

Progress

You can track the progress of multiple NSProgress objects using the child/parent relationship. You can create an NSProgress *overallProgress object and then call [overallProgress becomeCurrentWithPendingUnitCount:<unit size>]; before creating a new request and [overallProgress resignCurrent]; afterwards. Your overallProgress object will reflect the progress of all the children.

Baize