How to overlay one video on another in iOS?

Question

I am trying to crop an already taken video into a circle in iOS. How might I go about doing this. I know how I would do it with AVCaptureSession but I don't know to pass in an already taken video as an AVCaptureDevice? Is there a way to crop a video into a circle. I want to overlay it on top of another video so it has to have a transparent background as well. Thanks.

Gulden

Do you want to generate a file containing the cropped video, or do you just want to display it live on the screen clipped to a circle? – Kitten

I have two videos, and I want to put the circle video on top of the other videos where the circle video is transparent. – Gulden

But do you want to write the combined image to a file, or just display it on the screen? – Kitten

It is a video. I want to crop a video into a circle and put that video on top of another video. The video doesn't save transparency so I don't know how to combine these two videos. I want to then save the video – Gulden

@BlaneTownsend any chance you could help me through the code rob posted, Ive been trying to make some changes for what I want but can't quite get it? – Inaudible

Answer 1

I guess you want to produce something like this:

You don't want an AVCaptureSession, because you're not capturing video. You want an AVMutableComposition. You need to read the “Editing” section of the AV Foundation Programming Guide. Here's a summary of what you need to do:

1. Create the AVAsset objects for your videos and wait for them to load their tracks.

2. Create an AVMutableComposition.

3. Add a separate AVMutableCompositionTrack to the composition for each of the input videos. Make sure to assign explicit, different track IDs to each track. If you let the system pick, it will use track ID 1 for each and you won't be able to access both later in the compositor.

4. Create an AVMutableVideoComposition.

5. Create an AVMutableVideoCompositionInstruction.

6. For each input video, create an AVMutableVideoCompositionLayerInstruction and explicitly assign the track IDs you used back in step 3.

7. Set the AVMutableVideoCompositionInstruction's layerInstructions to the two layer instructions you created in step 6.

8. Set the AVMutableVideoComposition's instructions to the instruction you created in step 5.

9. Create a class that implements the AVVideoCompositing protocol. Set the customVideoCompositorClass of the video composition (created in step 4) to this custom class (e.g. videoComposition.customVideoCompositorClass = [CustomVideoCompositor class];).

10. In your custom compositor, get the input pixel buffers from the AVAsynchronousVideoCompositionRequest and use them to draw the composite frame (containing a background video frame overlaid by a circular chunk of the foreground video frame). You can do this however you want. I did it using Core Graphics because that's easy, but you'll probably want to use OpenGL (or maybe Metal) for efficiency in a production app. Be sure to specify kCVPixelBufferOpenGLESCompatibilityKey if you go with OpenGL.

11. Create an AVAssetExportSession using your composition from step 1.

12. Set the session's output URL and file type.

13. Set the session's videoComposition to the video composition from step 4.

14. Tell the session to exportAsynchronouslyWithCompletionHandler:. It will probably be slow!

You can find my test project in this github repository.

Kitten

Rob this is amazing thanks so much. @BlaneTownsend hey I was wondering how to get the circle video once its made onto the screen, in view controller.m? Thanks – Inaudible

@Inaudible The AV Foundation Programming Guide has a whole chapter on playing videos, and Apple has sample code and WWDC videos about it too. – Kitten

great thanks rob. I have got my videos working now, but I am having some trouble making the inner oval the correct size I want to encapture the right amount of the inner vid. I would really appreciate it if you could spare a bit of time and help me, I invited you to a chat or my skype is spennyfontein. Thanks man – Inaudible

rob I could start a bounty if you like, I just need some help with the size? – Inaudible

I cannot promise to help you, but it sounds like you should post your own question. – Kitten

If you wouldn't mind here is a link to the question #29449436 – Inaudible

rob any chance you could help, i keep looking over your code but can't quite get it – Inaudible

Apple provides some sample code that implements AVVideoCompositing using OpenGL: developer.apple.com/library/ios/samplecode/AVCustomEdit/… – Hanging

This is great. I'm attempting to modify it to work with videos of different lengths; specifically I'd like to loop the front video if it's shorter than the back video. Any thoughts on how to do that? All I can think of is to create a front asset that repeats until it is at least the length of the back asset, but perhaps there is a better way? – Rod

I suspect you could use -[AVMutableComposition insertTimeRange:ofTrack:atTime:error:] to duplicate the contents of your shorter track as necessary. I don't have time to experiment with it. – Kitten

@rob mayoff Do you think, we can implement this effect without using customVideoCompositorClass but only using [AVVideoCompositionCoreAnimationTool videoCompositionCoreAnimationToolWithPostProcessingAsVideoLayers]? – Sutphin

Answer 2

Swift Video overlay

You have next options:

1. Add extra overlay(image, animation) using AVMutableVideoComposition. animationTool. Actually it is a kind of post processing where extra overlay is added to your video

Add corners to video:

func applyCornersAsOverlayToComposition(
    composition: AVMutableVideoComposition,
    coverViewFrameSize: CGSize
) {
    //set up the parent layer
    let parentLayer = CALayer()
    
    //apply corners
    parentLayer.masksToBounds = true
    parentLayer.cornerRadius = CollagePresenter.containerViewCornerRadius
    
    let videoLayer = CALayer()
    parentLayer.frame = CGRectMake(0, 0, coverViewFrameSize.width, coverViewFrameSize.height)
    videoLayer.frame = CGRectMake(0, 0, coverViewFrameSize.width, coverViewFrameSize.height)
    
    //priority is important to make an overlay
    parentLayer.addSublayer(videoLayer)
    
    let animationTool = AVVideoCompositionCoreAnimationTool(postProcessingAsVideoLayer: videoLayer, in: parentLayer)
    composition.animationTool = animationTool
}

2. Overlap multiple videos using AVMutableVideoComposition.instructions and AVMutableVideoCompositionLayerInstruction. It allows you to transform video (rotate, translate, scale...) and combine multiple of them

private func foo(
    videoAsset1: AVURLAsset,
    videoAsset2: AVURLAsset
) {
    let composition = AVMutableComposition()
    
    let trackVideo1 = videoAsset1.tracks(withMediaType: .video)[0]
    let trackVideo2 = videoAsset2.tracks(withMediaType: .video)[0]
    
    let videoTrack1 = composition.addMutableTrack(
        withMediaType: .video,
        preferredTrackID: kCMPersistentTrackID_Invalid
    )!
    
    let videoTrack2 = composition.addMutableTrack(
        withMediaType: .video,
        preferredTrackID: kCMPersistentTrackID_Invalid
    )!
    
    try! videoTrack1.insertTimeRange(
        CMTimeRangeMake(start: CMTime.zero, duration: videoAsset1.duration),
        of: trackVideo1,
        at: CMTime.zero
    )
    
    try! videoTrack2.insertTimeRange(
        CMTimeRangeMake(start: CMTime.zero, duration: videoAsset2.duration),
        of: trackVideo1,
        at: CMTime.zero
    )
    
    let transform1 = CGAffineTransform(scaleX: 0.1, y: 0.1)
        .concatenating(CGAffineTransform(rotationAngle: 0))
        .concatenating(
            CGAffineTransformMakeTranslation(
                0,
                0
            )
        )
    
    let transform2 = CGAffineTransform(scaleX: 0.2, y: 0.2)
        .concatenating(CGAffineTransform(rotationAngle: 0))
        .concatenating(
            CGAffineTransformMakeTranslation(
                2,
                2
            )
        )
    
    let layerInstruction1 = AVMutableVideoCompositionLayerInstruction(assetTrack: videoTrack1)
    layerInstruction1.setTransform(
        transform1, at: CMTime.zero
    )
    
    let layerInstruction2 = AVMutableVideoCompositionLayerInstruction(assetTrack: videoTrack2)
    layerInstruction2.setTransform(
        transform2, at: CMTime.zero
    )
    
    let mainInstruction = AVMutableVideoCompositionInstruction()
    mainInstruction.backgroundColor = UIColor.yellow.cgColor
    
    //Max duration
    mainInstruction.timeRange = CMTimeRangeMake(
        start: CMTime.zero,
        duration: max(videoAsset1.duration, videoAsset2.duration)
    )
    
    mainInstruction.layerInstructions = [layerInstruction1, layerInstruction2]
    
    let videoComposition = AVMutableVideoComposition()
    videoComposition.renderSize = CGSize(width: 1920, height: 1080)
    videoComposition.frameDuration = CMTimeMake(value: 1, timescale: 30)
    
    videoComposition.instructions = [mainInstruction]
}

3. Overlap multiple videos with custom instructions using AVMutableVideoComposition.customVideoCompositorClass

let videoComposition = AVMutableVideoComposition()
videoComposition.renderSize = CGSize(width: 1920, height: 1080)
videoComposition.frameDuration = CMTimeMake(value: 1, timescale: 30)

//every frame processing
videoComposition.customVideoCompositorClass = CustomCompositor.self
let customOverlayInstruction = CustomOverlayInstruction(
    timeRange: CMTimeRangeMake(
        start: CMTime.zero,
        duration: videoAsset.duration
    ),
    videoEntries: videoEntries
)

videoComposition.instructions = [customOverlayInstruction]

Stroboscope