CIImage back to CMSampleBuffer

Question

I record video (.mp4 file) using AVAssetWriter with CMSampleBuffer data (from video, audio inputs).

While recording I want to process frames, I'm converting CMSampleBuffer to CIImage and processing it.

but how to update CMSampleBuffer with my new processed image buffer from CIImage?

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    if output == videoOutput {
       let imageBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!
       let ciimage: CIImage = CIImage(cvPixelBuffer: imageBuffer)
       ... // my code to process CIImage (for example add augmented reality)
       // but how to convert it back to CMSampleBuffer?
       // because AVAssetWriterInput to encode video/audio in file needs CMSampleBuffer
       ...
    } 
    ...
}

Shuma

Answer 1

You need to render your CIImage into a CVPixelBuffer, using CIContext's render(_:to:bounds:colorSpace:) method.

Then you can create a CMSampleBuffer from the CVPixelBuffer using e.g. CMSampleBufferCreateReadyWithImageBuffer(_:_:_:_:_:)

You may need to use a pool of CVPixelBuffer for efficiency reasons, an example of this is shown in Apple's AVCamPhotoFilter sample code. In particular, see the RosyCIRenderer class.

Also see this answer which may help you Applying a CIFilter to a Video File and Saving it

Mann

yes I found answer myself already (I already gave a link), this question can be closed - #49066695 and performance sucks that's why I asked different question about best way to add text/ better performance - #49066695 – Shuma