I'm using WebRTC to create a peer to peer connection for sharing screen and audio. I'm capturing the screen using ReplayKit, which generates CMSampleBufferRef; using that I can create RTCVideoFrame.
To get the CMSampleBufferRef I'm using:
[[RPScreenRecorder sharedRecorder] startCaptureWithHandler:^(CMSampleBufferRef _Nonnull sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable error)
Up until this point, everything works perfectly.
The problem arises when I start sending the app to background and come back several times; then ReplayKit stops calling his capture handler.
This only happens if I send the CMSampleBufferRef to WebRTC, so it's clear that the ReplayKit issue is related to WebRTC. If I delete this line from the code, the issue doesn't happen (clearly WebRTC wouldn't work, though).
[self->source capturer:self->capturer didCaptureVideoFrame:videoFrame];
The only way I can get it to work again is to restart the device. Even killing the app and restarting doesn't work.
This is how I create RTCVideoTrack in my view controller:
- (RTCVideoTrack *)createLocalVideoTrack {
self->source = [_factory videoSource];
self->capturer = [[RTCVideoCapturer alloc] initWithDelegate:self->source];
[self->source adaptOutputFormatToWidth:441 height:736 fps:15];
return [_factory videoTrackWithSource:self->source trackId:@"ARDAMSv0"];
}
Here is how I convert the CMSampleBufferRef to RTCVideoFrame and send to WebRTC:
- (void)didCaptureSampleBuffer:(CMSampleBufferRef)sampleBuffer {
if (CMSampleBufferGetNumSamples(sampleBuffer) != 1 || !CMSampleBufferIsValid(sampleBuffer) ||
!CMSampleBufferDataIsReady(sampleBuffer)) {
return;
}
CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
if (pixelBuffer == nil) {
return;
}
RTCCVPixelBuffer *rtcPixelBuffer = [[RTCCVPixelBuffer alloc] initWithPixelBuffer:pixelBuffer];
int64_t timeStampNs =
CMTimeGetSeconds(CMSampleBufferGetPresentationTimeStamp(sampleBuffer)) * NSEC_PER_SEC;
RTCVideoFrame *videoFrame = [[RTCVideoFrame alloc] initWithBuffer:rtcPixelBuffer rotation:RTCVideoRotation_0 timeStampNs:timeStampNs];
[self->source capturer:self->capturer didCaptureVideoFrame:videoFrame];
}