Transform not working in AVMutableVideoComposition while exporting

Question

My goal is to compose a set of clips recorded from the camera and export them at a certain preferred size. Of course, the video orientation needs to be rotated before exporting.

I'm doing this by composing an AVMutableComposition from an array of video clips, stored in avAssets below. I am able to compose them fine, and export it. However, the rotation transform I am setting on the AVMutableVideoComposition is not being honored. If I use the same transform and set it on the preferredTransform property of the video track, then it works. In both cases, the video renderSize is not being honored. It's as if the exporter ignores the videoComposition completely. Any ideas what could be happening?

I do have an AVCaptureSession running, but I turned it off before exporting and that didn't make any difference. I am fairly new to iOS programming, so it could be I'm missing something basic. :)

My code:

-(void) finalRecord{
NSError *error = nil;

AVMutableComposition *composition = [AVMutableComposition composition];

AVMutableCompositionTrack *compositionVideoTrack = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
NSLog(@"Video track id is %d", [compositionVideoTrack trackID]);

AVMutableCompositionTrack *compositionAudioTrack = [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];

// avAssets hold the video clips to be composited
int pieces = [avAssets count];

CGAffineTransform transform = CGAffineTransformMakeRotation( M_PI_2);
//  [compositionVideoTrack setPreferredTransform:transform];

for (int i = 0; i<pieces; i++) {

    AVURLAsset *sourceAsset = [avAssets objectAtIndex:i];

    AVAssetTrack *sourceVideoTrack = [[sourceAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
    AVAssetTrack *sourceAudioTrack = [[sourceAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];

    [timeRanges addObject:[NSValue valueWithCMTimeRange:CMTimeRangeMake(kCMTimeZero, sourceAsset.duration)]];
    [videoTracks addObject:sourceVideoTrack];
    [audioTracks addObject:sourceAudioTrack];
}

[compositionVideoTrack insertTimeRanges:timeRanges ofTracks:videoTracks atTime:kCMTimeZero error:&error];
[compositionAudioTrack insertTimeRanges:timeRanges ofTracks:audioTracks atTime:kCMTimeZero error:&error];

AVMutableVideoCompositionInstruction *vtemp = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
vtemp.timeRange = CMTimeRangeMake(kCMTimeZero, [composition duration]);
NSLog(@"\nInstruction vtemp's time range is %f %f", CMTimeGetSeconds( vtemp.timeRange.start),
      CMTimeGetSeconds(vtemp.timeRange.duration));

// Also tried videoCompositionLayerInstructionWithAssetTrack:compositionVideoTrack    
AVMutableVideoCompositionLayerInstruction *vLayerInstruction = [AVMutableVideoCompositionLayerInstruction
                                                               videoCompositionLayerInstructionWithAssetTrack:composition.tracks[0]];
[vLayerInstruction setTransform:transform atTime:kCMTimeZero];
vtemp.layerInstructions = @[vLayerInstruction];

AVMutableVideoComposition *videoComposition = [AVMutableVideoComposition videoComposition];
videoComposition.renderSize = CGSizeMake(320.0, 240.0);
videoComposition.frameDuration = CMTimeMake(1,30);

videoComposition.instructions = @[vtemp];

AVAssetExportSession *exporter = [[AVAssetExportSession alloc] initWithAsset:composition presetName:gVideoExportQuality];

NSParameterAssert(exporter != nil);
exporter.videoComposition = videoComposition;
exporter.outputFileType = AVFileTypeQuickTimeMovie;

NSString *rootName = [[self captureManager] tempFileRoot];

NSString *temp = [NSString stringWithFormat:@"%@%@.mov", NSTemporaryDirectory(), rootName];
exporter.outputURL = [NSURL fileURLWithPath:temp ];

[exporter exportAsynchronouslyWithCompletionHandler:^{
    switch ([exporter status]) {
        case AVAssetExportSessionStatusFailed:
            NSLog(@"Export failed: %@", [exporter error]);
            break;
        case AVAssetExportSessionStatusCancelled:
            NSLog(@"Export canceled");
            break;
        case AVAssetExportSessionStatusCompleted:
            NSLog(@"Export successfully");
            [self exportFile:exporter.outputURL];
            [self.delegate recordingEndedWithFile:exporter.outputURL];
            isExporting = FALSE;
            [[[self captureManager] session] startRunning];
            break;
        default:
            break;
    }
    if (exporter.status != AVAssetExportSessionStatusCompleted){
        NSLog(@"Retry export");
    }
}];

}

Milkman

Answer 1

Ok figured it out and posting here to help other people not waste the time that I did.

The issue is that if you use AVAssetExportPresetPassthrough on an AVExportSession, then the exporter will ignore the video composition instructions. I was expecting it to at least honor the video composition instructions while passing through the format etc. but apparently that isn't how it works. After I have filled a documentation bug, you can find it in Technical Q&A.

Milkman

So what was the solution? – Mcnalley

To solve it, just use another setting other than AVAssetExportPresetPassthrough that makes sense for your app. – Milkman

This is documented here : developer.apple.com/library/ios/qa/qa1744/_index.html – Lovelorn

I filed a bug with Apple when I first posted here - glad to see that they modified their doc. – Milkman

Also, they added message to the console: WARNING: AVAssetExportSession will ignore videoComposition when using AVAssetExportPresetPassthrough preset. – Faltboat

Changing to AVAssetExportPresetMediumQuality doesn't seem to help here. It doesn't seem to rotate the video, but it clears the rotation metadata info. – Wareing

Perhaps something changed with iOS8? – Milkman

I'm not using AVAssetExportPresetPassthrough, and still the rotation isn't working. I recently started using Metal, and I'm afraid it's somehow relating to that. – Dauntless

Answer 2

The solution if you want to use AVAssetExportPresetPassthrough:

compositionVideoTrack.preferredTransform = transform

more info here: https://developer.apple.com/library/archive/qa/qa1744/_index.html

Otherwise, if you specify the AVAssetExportPresetPassthrough export option to let all tracks pass through but you still want to set a transform on the composition, set the preferredTransform property on the composition tracks as described above.

Chun