Simultaneous AVCaptureVideoDataOutput and AVCaptureMovieFileOutput

Question

I need to be able to have AVCaptureVideoDataOutput and AVCaptureMovieFileOutput working at the same time. The below code works, however, the video recording does not. The didFinishRecordingToOutputFileAtURL delegate is called directly after startRecordingToOutputFileURL is called. Now if i remove AVCaptureVideoDataOutput from the AVCaptureSession by simply commenting out the line:

[captureSession addOutput:captureDataOutput];

The video recording works but then the SampleBufferDelegate is not called (which i need).

How can i go about having AVCaptureVideoDataOutput and AVCaptureMovieFileOutput working simultaneously.

- (void)initCapture {
 AVCaptureDeviceInput *captureInput = [AVCaptureDeviceInput deviceInputWithDevice:[AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo] error:NULL]; 

 captureDataOutput = [[AVCaptureVideoDataOutput alloc] init]; 
 [captureDataOutput setSampleBufferDelegate:self queue:dispatch_get_main_queue()]; 

 m_captureFileOutput = [[AVCaptureMovieFileOutput alloc] init];

 NSString* key = (NSString*)kCVPixelBufferPixelFormatTypeKey; 
 NSNumber* value = [NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA]; 
 NSDictionary* videoSettings = [NSDictionary dictionaryWithObject:value forKey:key]; 

 [captureDataOutput setVideoSettings:videoSettings]; 

 captureSession = [[AVCaptureSession alloc] init]; 

 [captureSession addInput:captureInput];
 [captureSession addOutput:m_captureFileOutput]; 
 [captureSession addOutput:captureDataOutput]; 

 [captureSession beginConfiguration]; 
 [captureSession setSessionPreset:AVCaptureSessionPresetLow]; 
 [captureSession commitConfiguration]; 

 [self performSelector:@selector(startRecording) withObject:nil afterDelay:10.0];
 [self performSelector:@selector(stopRecording) withObject:nil afterDelay:15.0];

 [captureSession startRunning];
}


- (void) startRecording
{
    [m_captureFileOutput startRecordingToOutputFileURL:[self tempFileURL] recordingDelegate:self];

}

- (void) stopRecording
{
    if([m_captureFileOutput isRecording])
 [m_captureFileOutput stopRecording];

}


- (NSURL *) tempFileURL
{
 NSString *outputPath = [[NSString alloc] initWithFormat:@"%@%@", NSTemporaryDirectory(), @"camera.mov"];
 NSURL *outputURL = [[NSURL alloc] initFileURLWithPath:outputPath];
 NSFileManager *fileManager = [NSFileManager defaultManager];
 if ([fileManager fileExistsAtPath:outputPath]) {
  [[NSFileManager defaultManager] removeItemAtPath:outputPath error:nil];
 [outputPath release];
 return [outputURL autorelease];
}



- (void)captureOutput:(AVCaptureFileOutput *)captureOutput didStartRecordingToOutputFileAtURL:(NSURL *)fileURL fromConnections:(NSArray *)connections
{
 NSLog(@"start record video");
}

- (void)captureOutput:(AVCaptureFileOutput *)captureOutput didFinishRecordingToOutputFileAtURL:(NSURL *)outputFileURL fromConnections:(NSArray *)connections error:(NSError *)error
{
 NSLog(@"end record");
}


- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection 
{ 
      // do stuff with sampleBuffer
}

I should add i am getting the error:

Error Domain=NSOSStatusErrorDomain Code=-12780 "The operation couldn’t be completed. (OSStatus error -12780.)" UserInfo=0x23fcd0 {AVErrorRecordingSuccessfullyFinishedKey=false}

from

- (void)captureOutput:(AVCaptureFileOutput *)captureOutput didFinishRecordingToOutputFileAtURL:(NSURL *)outputFileURL fromConnections:(NSArray *)connections error:(NSError *)error

Cheers

Decomposed

There is a proven working way to do this. More in this question: #4944583 – Hurried

Answer 1

I have contacted an engineer at Apple's support and he told me that simultaneous AVCaptureVideoDataOutput + AVCaptureMovieFileOutput use is not supported. I don't know if they will support it in the future, but he used the word "not supported at this time".

I encourage you to fill a bug report / feature request on this, as I did (bugreport.apple.com), as they measure how hard people want something and we perhaps can see this in a near future.

Toothpick

Be sure to see Oded's comment above with a working solution: #4944583 – Meniscus

@StanJames is it still the case today where oded's workaround is the best approach to using AVCaptureVideoDataOutput + AVCaptureMovieFileOutput simultaneously? – Tangy

@StanJames if the goal is to create two videos simultaneously could you use two AVAssetWriters (with only AVCaptureVideoDataOutput) instead of both AVCaptureVideoDataOutput and AVCaptureMovieFileOutput? – Tangy

Does anyone have an updates on whether this is still impossible or not? – Retardation

@SpaceDog Could you provide a proof of this? – Shroudlaid

@SashaKid - This was 6 years ago. At the time I have opened a paid technical incident to ask that and this was what they answered me, but like I said, that was 6 years ago. Perhaps it is possible to do that today, I don't know. – Toothpick

@SpaceDog I see that it's still not possible to have both AVCaptureVideoDataOutput + AVCaptureMovieFileOutput. My delegate method AVCaptureVideoDataOutputSampleBufferDelegate not called :-( – Shroudlaid

To be honest, that's kind of ridiculous. There are so many situations where you'd need both and 7 years later it's still not possible. – Austin

Apple never fixes anything that is important, unless it blows on them by the press. They fix what commercially makes sense for them. I have bugs there since 2008 that were never fixed and will ever be fixed. – Toothpick

03.2020 still nothing T_T – Edinburgh

Seems to be working on iOS 16: iOS 15 nope, iOS 16 yes, iOS 17 yes – Glomerule

Answer 2

Still 9 years later Apple apparently does not seem to want this to work together.

But you can easily work with AVAssetWriter.

You can't use AVCaptureVideoDataOutput and AVCaptureMovieFileOutput on the same time. But you can use AVCaptureVideoDataOutput and analyse or modify on the data, then use AVAsseWriter to write the frames to a file.

Source: https://developer.apple.com/forums/thread/98113

How to save video with output using AVAssetWriter:

Save AVCaptureVideoDataOutput to movie file using AVAssetWriter in Swift

Hewett

Answer 3

Although you cannot use AVCaptureVideoDataOutput, you can use AVCaptureVideoPreviewLayer simultaneously with AVCaptureMovieFileOutput. See the "AVCam" example on Apple's Website.

In Xamarin.iOS, the code looks like this:

var session = new AVCaptureSession();

var camera = AVCaptureDevice.DefaultDeviceWithMediaType(AVMediaType.Video);
var  mic = AVCaptureDevice.DefaultDeviceWithMediaType(AVMediaType.Audio);
if(camera == null || mic == null){
    throw new Exception("Can't find devices");
}

if(session.CanAddInput(camera)){
    session.AddInput(camera);
}
if(session.CanAddInput(mic)){
   session.AddInput(mic);
}

var layer = new AVCaptureVideoPreviewLayer(session);
layer.LayerVideoGravity = AVLayerVideoGravity.ResizeAspectFill;
layer.VideoGravity = AVCaptureVideoPreviewLayer.GravityResizeAspectFill;

cameraView = new UIView();
cameraView.Layer.AddSublayer(layer);

var filePath = System.IO.Path.Combine( Path.GetTempPath(), "temporary.mov");
var fileUrl = NSUrl.FromFilename( filePath );

var movieFileOutput = new AVCaptureMovieFileOutput();
var recordingDelegate = new MyRecordingDelegate();
session.AddOutput(movieFileOutput);

movieFileOutput.StartRecordingToOutputFile( fileUrl, recordingDelegate);

Berneta

Answer 4

XCode 14.1 is already support it.

• XCode 13.4: Not works
• XCode 14.1: works

Wardell

This is not true. I upgraded my XCode to 14.1 today and still, as soon as you add AVCaptureMovieFileOutput to the capture session, AVCaptureVideoDataOutputSampleBufferDelegate stops receiving sample buffers. – Lott

It still works on XCode 14.1 my side. Please try to move AVCaptureMovieFileOutput to before AVCaptureMovieDataOutput and try again. – Wardell

Nope, still doesn't work, can't get any sample buffers. – Lott

@Ashkan Sarlak Step1: comment out AVCaptureMovieFileOutput then AVCaptureMovieDataOutput works?. Step2: comment out AVCaptureMovieDataOutput then AVCaptureMovieFileOutput works?. Please make sure you did add delegate function for captureOutput. – Wardell

1. yes. 2. yes and yes everything is in place. Actually I just did it using AVAssetWriter trick and right now don't care about this approach anymore. Thank you for this discussion. – Lott

Of course AVAssetWriter alway works. But I don't know why AVCaptureMovieFileOutput and AVCaptureMovieDataOutput not worked your side. If you are using AVAssetWriter with audio, please take care blank frames at first and last video because sometime Audio Buffer coming first and last. – Wardell

That was a good advice, seems like audio buffer comes first and there'll be some blank frames at the start, thanks. – Lott