Weird behaviour of AVMutableComposition freezing while using AVMutableVideoComposition

Question

I am trying to merge multiple videos using AVMutableComposition. The problem I face is that whenever I try to add any AVMutableVideoComposition for applying any instructions, my playback freezes in AVPlayer at exact 6 seconds duration.

Another interesting thing is that it plays fine if I play it in Photos app of iPad after exporting using same videoComposition. So why does it freezes in AVPlayer at 6 seconds?

Code:

AVMutableComposition *mutableComposition = [AVMutableComposition composition];

    AVMutableCompositionTrack *videoCompositionTrack = [mutableComposition addMutableTrackWithMediaType:AVMediaTypeVideo
                                                                                        preferredTrackID:kCMPersistentTrackID_Invalid];

    AVMutableCompositionTrack *audioCompositionTrack = [mutableComposition addMutableTrackWithMediaType:AVMediaTypeAudio
                                                                                       preferredTrackID:kCMPersistentTrackID_Invalid];

    for (AVURLAsset *asset in assets)
    {
            AVAssetTrack *assetTrack;
            assetTrack = [[asset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];
            AVAssetTrack *audioAssetTrack = [asset tracksWithMediaType:AVMediaTypeAudio].firstObject;


            NSError *error;
            [videoCompositionTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, assetTrack.timeRange.duration )
                                                   ofTrack:assetTrack
                                                    atTime:time
                                                     error:&error];

            if (error) {
                NSLog(@"asset url :: %@",assetTrack.asset);
                NSLog(@"Error1 - %@", error.debugDescription);
            }

            [audioCompositionTrack insertTimeRange:CMTimeRangeMake(kCMTimeZero, audioAssetTrack.timeRange.duration)
                                               ofTrack:audioAssetTrack
                                                atTime:time
                                                 error:&error];

            if (error) {
                NSLog(@"Error2 - %@", error.debugDescription);
            }

            time = CMTimeAdd(time, assetTrack.timeRange.duration);

            if (CGSizeEqualToSize(size, CGSizeZero)) {
                size = assetTrack.naturalSize;;
            }
        }

        AVMutableVideoCompositionInstruction *mainInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
        AVMutableVideoCompositionLayerInstruction *videoTrackLayerInstruction =  [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:videoCompositionTrack];

        mainInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, time);
        mainInstruction.layerInstructions = [NSArray arrayWithObjects:videoTrackLayerInstruction, nil];
        AVMutableVideoComposition *mainCompositionInst = [AVMutableVideoComposition videoComposition];
        mainCompositionInst.instructions = [NSArray arrayWithObject:mainInstruction];
        mainCompositionInst.frameDuration = CMTimeMake(1, 30);
        mainCompositionInst.renderSize = size;

        pi = [AVPlayerItem playerItemWithAsset:mutableComposition];
        pi.videoComposition = mainCompositionInst;

Also, I know problem is mostly videoComposition because if I remove the videoComposition, then it plays fine on AVPlayer.

UPDATE 1: I just found out that when it freezes after 6 seconds, if I drag the slider back or forward(i.e. use seekToTime), it starts playing normally again without any further freeze.

Also audio keeps on playing fine even when video is frozen.

UPDATE 2: If I just go ahead and export it using AVAssetExportSession with same AVMutableComposition, and load asset from of exported video, it works fine. So its just when I play AVMutableComposition directly, problem arises.

Tamarisk

I had same issue, any luck? – Tomika

Again; this still happens. Can PLEASE someone react here? – Glutinous

@Julian Not able to solve this up to now. Ended up settling with exporting it which involves waiting. No attention to problem after offering bounty also. Keep upvoting and hope someone notices it eventually. – Tamarisk

@Julian Please check my answer. – Popcorn

I just fixed the issue, like 30 mins ago, thanks either way. I actually had a different problem. I created instructions for the asset track instead the composition track... Silly one, but still took me like 3 weeks to find the right combination – Glutinous

Answer 1

Finally, I got the solution to fix this.

You should play after playerItem's status is changed to .ReadyToPlay.

Please see as below.

func startVideoPlayer() {
    let playerItem = AVPlayerItem(asset: self.composition!)
    playerItem.videoComposition = self.videoComposition!

    let player = AVPlayer(playerItem: playerItem)
    player.actionAtItemEnd = .None

    videoPlayerLayer = AVPlayerLayer(player: player)
    videoPlayerLayer!.frame = self.bounds

    /* add playerItem's observer */
    player.addObserver(self, forKeyPath: "player.currentItem.status", options: .New, context: nil)

    NSNotificationCenter.defaultCenter().addObserver(self, selector: "playerItemDidReachEnd:", name: AVPlayerItemDidPlayToEndTimeNotification, object: playerItem);

    self.layer.addSublayer(videoPlayerLayer!)
}

override func observeValueForKeyPath(keyPath: String?, ofObject object: AnyObject?, change: [String : AnyObject]?, context: UnsafeMutablePointer<Void>) {
    if keyPath != nil && keyPath! == "player.currentItem.status" {
        if let newValue = change?[NSKeyValueChangeNewKey] {
            if AVPlayerStatus(rawValue: newValue as! Int) == .ReadyToPlay {
                playVideo() /* play after status is changed to .ReadyToPlay */
            }
        }
    } else {
        super.observeValueForKeyPath(keyPath, ofObject: object, change: change, context: context)
    }
}    

func playerItemDidReachEnd(notification: NSNotification) {
    let playerItem = notification.object as! AVPlayerItem
    playerItem.seekToTime(kCMTimeZero)

    playVideo()
} 

func playVideo() {
    videoPlayerLayer?.player!.play()
}       

Popcorn