Record video with AVAssetWriter: first frames are black

Question

I am recording video (the user also can switch to audio only) with AVAssetWriter. I start the recording when the app is launched. But the first frames are black (or very dark). This also happens when I switch from audio to video. It feels like the AVAssetWriter and/or AVAssetWriterInput are not yet ready to record. How can I avoid this?

I don't know if this is a useful info but I also use a GLKView to display the video.

func start_new_record(){
    do{
        try self.file_writer=AVAssetWriter(url: self.file_url!, fileType: AVFileTypeMPEG4)
        if video_on{
            if file_writer.canAdd(video_writer){
                file_writer.add(video_writer)
            }
        }
        if file_writer.canAdd(audio_writer){
            file_writer.add(audio_writer)
        }
    }catch let e as NSError{
        print(e)
    }
}

func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!){
    guard is_recording else{
        return
    }

    guard CMSampleBufferDataIsReady(sampleBuffer) else{
        print("data not ready")
        return
    }

    guard let w=file_writer else{
        print("video writer nil")
        return
    }

    if w.status == .unknown && start_recording_time==nil{
        if (video_on && captureOutput==video_output) || (!video_on && captureOutput==audio_output){
            print("START RECORDING")
            file_writer?.startWriting()
            start_recording_time=CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
            file_writer?.startSession(atSourceTime: start_recording_time!)
        }else{
            return
        }
    }

    if w.status == .failed{
        print("failed /", w.error ?? "")
        return
    }

    if captureOutput==audio_output{
        if audio_writer.isReadyForMoreMediaData{
            if !video_on || (video_on && video_written){
                audio_writer.append(sampleBuffer)
                //print("write audio")
            }
        }else{
            print("audio writer not ready")
        }
    }else if video_output != nil && captureOutput==video_output{
        if video_writer.isReadyForMoreMediaData{
            video_writer.append(sampleBuffer)
            if !video_written{
                print("added 1st video frame")
                video_written=true
            }
        }else{
            print("video writer not ready")
        }
    }
}

Yurev

For me the solution was to call file_writer?.startWriting() and on the next runloop call file_writer?.startSession(atSourceTime: start_recording_time!). Never had issues ever since. – Overweigh

Answer 1

Ok, stupid mistake...

When launching the app, I init my AVCaptureSession, add inputs, outputs, etc. And I was just calling start_new_record a bit too soon, just before commitConfiguration was called on my capture session.

At least my code might be useful to some people.

Yurev

I tried this, but I still get black frames. What else to try? – Merv

@VaddadiKartick I'm sorry I don't know. I'd say you should check how you configure it, like me. Do you init everything on the same queue? Did you try not to start the recording when the app is launching, like me, but to start it manually? – Yurev

Answer 2

SWIFT 4

SOLUTION #1:

I resolved this by calling file_writer?.startWriting() as soon as possible upon launching the app. Then when you want to start recording, do the file_writer?.startSession(atSourceTime:...).

When you are done recording and call finishRecording, when you get the callback that says that's complete, set up a new writing session again.

SOLUTION #2:

I resolved this by adding half a second to the starting time when calling AVAssetWriter.startSession, like this:

start_recording_time = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
let startingTimeDelay = CMTimeMakeWithSeconds(0.5, 1000000000)
let startTimeToUse = CMTimeAdd(start_recording_time!, startingTimeDelay)

file_writer?.startSession(atSourceTime: startTimeToUse)

SOLUTION #3:

A better solution here is to record the timestamp of the first frame you receive and decide to write, and then start your session with that. Then you don't need any delay:

//Initialization, elsewhere:

var is_session_started = false
var videoStartingTimestamp = CMTime.invalid

// In code where you receive frames that you plan to write:

if (!is_session_started) {
    // Start writing at the timestamp of our earliest sample
    videoStartingTimestamp = currentTimestamp
    print ("First video sample received: Starting avAssetWriter Session: \(videoStartingTimestamp)")
    avAssetWriter?.startSession(atSourceTime: videoStartingTimestamp)

    is_session_started = true
}

// add the current frame
pixelBufferAdapter?.append(myPixelBuffer, withPresentationTime: currentTimestamp)

Rozina

I used following delay 1/10th of second. let startingTimeDelay = CMTimeMakeWithSeconds(1, 10) – Adenovirus

I combine the first and third solution, and it still gives me a delay, and black frames in the beginning. – Tillie

I tried all the solutions but still getting black frames at the begining. Anybody found a solution? – Rosenkranz

Answer 3

This is for future users...

None of the above worked for me and then I tried changing the camera preset to medium which worked fine

Mccullers