AVFoundation AVMutableComposition - video with audio - freezes video

Question

I have a recorded video with sound. When adding that to an AVMutableComposition and then export it, the video will freeze when the sound is playing. What am I missing?

I have my two tracks:

_compositionVideoTrack = [self.composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
_compositionAudioTrack = [self.composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];

Then create the tracks:

AVAssetTrack *clipVideoTrack = [[sourceAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
AVAssetTrack *clipAudioTrack = [[sourceAsset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0];

Then I insert the tracks:

[_compositionVideoTrack insertTimeRange:editRange ofTrack:clipVideoTrack atTime:self.composition.duration error:&editError];
[_compositionAudioTrack insertTimeRange:editRange ofTrack:clipAudioTrack atTime:self.composition.duration error:&editError];

Finally I export it:

AVAssetExportSession *exportSession = [[AVAssetExportSession alloc] initWithAsset:urlAsset presetName:AVAssetExportPresetPassthrough];
exportSession.outputFileType = AVFileTypeQuickTimeMovie;

And the file is .mp4. Could it be the file format?

Limpid

Does the video freeze on a particular frame each time? It could be that the time range is incorrect for the video track. You are inserting the time range from your {clipVideoTrack} and {clipAudioTrack} to the composition track beginning at the composition's duration. – Tully

The video starts freezing when the sound starts. The audio and video can't be played at the same time. Yes, I want all clips added to the comp to be after the previous one, that why I add it to the duration of the comp. – Limpid

Without seeing more of your code, it's hard to say. My guess is that there's something wrong with the order of your AVMutableVideoCompositionLayerInstructions in the AVMutableVideoCompositionInstruction at the time when the sound starts. – Tully

What solution you found to this issue? Or have you found one? – Flume

Answer 1

Try Exporting using below

AVAssetExportSession *exportSession = [[AVAssetExportSession alloc] initWithAsset:self.inputAsset presetName:AVAssetExportPresetLowQuality];
    exportSession.outputFileType = AVFileTypeQuickTimeMovie;

Ramberg

I've tried changing AVAssetExportPresetPassthrough to AVAssetExportPresetMediumQuality. Didn't work either. – Limpid

Did you saved your video ?? or you have just exported ?? – Ramberg

Answer 2

You ca try to do like this:

AVMutableComposition *mutableComposition = [AVMutableComposition composition];
AVMutableCompositionTrack *videoCompositionTrack = [mutableComposition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
[videoCompositionTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, asset.duration) ofTrack:clipVideoTrack atTime:kCMTimeZero error:nil];
AVMutableCompositionTrack *audioCompositionTrack = [mutableComposition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];
[audioCompositionTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, asset.duration) ofTrack:[[asset tracksWithMediaType:AVMediaTypeAudio] objectAtIndex:0] atTime:kCMTimeZero error:nil];

and then init AVAssetExportSession with mutable composition

AVAssetExportSession* exporter = [[AVAssetExportSession alloc] initWithAsset:mutableComposition presetName:AVAssetExportPresetHighestQuality];

hope this will help you

Gambetta

Answer 3

I think you should add AVMutableVideoCompositionInstruction

Intercut

What should I add to the instruction? – Limpid