AVFoundation Crash on Exporting Video With Text Layer

Question

I'm developing a video editing app for iOS on my spare time.

I just resumed work on it after several weeks of attending other rpojects, and -even though I haven't made any significant changes to the code- now it crashes everytime I try to export my video composition.

I checked out and built the exact same commit that I successfully uploaded to TestFlight back then (and it was working on several devices without crashing), so perhaps it is an issue with the latest Xcode / iOS SDK that I hve updated since then?

The code crashes on _xpc_api_misuse, on a thread:

com.apple.coremedia.basicvideocompositor.output

Debug Navigator:

At the time of the crash, there are 70+ threads on the debug navigator, so perhaps something is wrong and the app is using too many threads (never seen these many).

---

My app overlays a 'watermark' on exported video using a text layer. After playing around, I discovered that the crash can be averted if I comment-out the watermark code:

    guard let exporter = AVAssetExportSession(asset: composition, presetName: AVAssetExportPresetHighestQuality) else {
        return failure(ProjectError.failedToCreateExportSession)
    }
    guard let documents = try? FileManager.default.url(for: .documentDirectory, in: .userDomainMask, appropriateFor: nil, create: true) else {
        return failure(ProjectError.temporaryOutputDirectoryNotFound)
    }
    let dateFormatter = DateFormatter()
    dateFormatter.dateFormat = "yyyy-MM-dd_HHmmss"
    let fileName = dateFormatter.string(from: Date())
    let fileExtension = "mov"
    let fileURL = documents.appendingPathComponent(fileName).appendingPathExtension(fileExtension)
    exporter.outputURL = fileURL

    exporter.outputFileType = AVFileType.mov
    exporter.shouldOptimizeForNetworkUse = true // check if needed

    // OFFENDING BLOCK (commenting out averts crash)
    if addWaterMark {
        let frame = CGRect(origin: .zero, size: videoComposition.renderSize)
        let watermark = WatermarkLayer(frame: frame)
        let parentLayer = CALayer()
        let videoLayer = CALayer()

        parentLayer.frame = frame
        videoLayer.frame = frame
        parentLayer.addSublayer(videoLayer)
        parentLayer.addSublayer(watermark)
        videoComposition.animationTool = AVVideoCompositionCoreAnimationTool(postProcessingAsVideoLayer: videoLayer, in: parentLayer)
    }
    // END OF OFFENDING BLOCK

    exporter.videoComposition = videoComposition

    exporter.exportAsynchronously {
    // etc.

The code for the watermark layer is:

class WatermarkLayer: CATextLayer {

    private let defaultFontSize: CGFloat = 48

    private let rightMargin: CGFloat = 10
    private let bottomMargin: CGFloat = 10

    init(frame: CGRect) {
        super.init()
        guard let appName = Bundle.main.infoDictionary?["CFBundleName"] as? String else {
            fatalError("!!!")
        }
        self.foregroundColor = CGColor.srgb(r: 255, g: 255, b: 255, a: 0.5)
        self.backgroundColor = CGColor.clear
        self.string = String(format: String.watermarkFormat, appName)
        self.font = CTFontCreateWithName(String.watermarkFontName as CFString, defaultFontSize, nil)
        self.fontSize = defaultFontSize
        self.shadowOpacity = 0.75
        self.alignmentMode = .right
        self.frame = frame
    }

    required init?(coder: NSCoder) {
        fatalError("init(coder:) has not been implemented. Use init(frame:) instead.")
    }

    override func draw(in ctx: CGContext) {
        let height = self.bounds.size.height
        let fontSize = self.fontSize
        let yDiff = (height-fontSize) - fontSize/10 - bottomMargin // Bottom (minus margin)

        ctx.saveGState()
        ctx.translateBy(x: -rightMargin, y: yDiff)
        super.draw(in: ctx)
        ctx.restoreGState()
    }
}

Any ideas what could be happening?

Perhaps my code is doing something wrong that somewhow 'got a pass' in a previous SDK due to some Apple bug that got fixed or an implementation 'hole' that got plugged?

---

UPDATE: I downloaded Ray Wenderlich's sample project for video wediting and tried to add 'subtitles' to a video (I had to tweak the too-old project so that it would compile under Xcode 11).

Lo and behold, it crashes in the exact same way.

---

UPDATE 2: I now tried on the device (iPhone 8 running the latest iOS 13.5) and it works, no crash. The Simulators for iOS 13.5 do crash however. When I originally posted the question (iOS 13.4?), I'm sure it was both Crashing on device and Simulator.

I am downloading the iOS 12.0 Simulators to check, but it's still a few gigabytes away...

Fred

I have also posted a question on Apple's forums (forums.developer.apple.com/message/422504#422504) so that it can gather some tumbleweeds. – Fred

Did you solve your issue? I am getting same issue. – Kweilin

@gstream79 Haven’t touched the code lately, busy on my day job. See “Update 2” above for the latest developments. – Fred

Getting crash on both device, simulator for iOS 13.4, 13.5 – Kweilin

Answer 1

Meet same issues, but on Simulator (Xcode 12.4 (12D4e)) only.

After some research, I found this crash is lead by AVVideoCompositionCoreAnimationTool's

+videoCompositionCoreAnimationToolWithPostProcessingAsVideoLayer:inLayer:

And I fixed it by replacing it w/ one below (but we need to handle instruction.layerInstructions in this way):

+videoCompositionCoreAnimationToolWithAdditionalLayer:asTrackID:

---

Below is a sample code works on both real device & simulator (as the OP didn't tag Swift explicitly, I'll just copy my Objective-C sample here):

...

// Prepare watermark layer
CALayer *watermarkLayer = ...;
CMPersistentTrackID watermarkLayerTrackID = [asset unusedTrackID];
// !!! NOTE#01: Use as additional layer here instead of animation layer.
videoComposition.animationTool = [AVVideoCompositionCoreAnimationTool videoCompositionCoreAnimationToolWithAdditionalLayer:watermarkLayer asTrackID:watermarkLayerTrackID];
  
// Create video composition instruction
AVMutableVideoCompositionInstruction *instruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
instruction.timeRange = CMTimeRangeMake(kCMTimeZero, asset.duration);

// - Watermark layer instruction
// !!! NOTE#02: Make this instruction track watermark layer by the `trackID`.
AVMutableVideoCompositionLayerInstruction *watermarkLayerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstruction];
watermarkLayerInstruction.trackID = watermarkLayerTrackID;

// - Video track layer instruction
AVAssetTrack *videoTrack = [asset tracksWithMediaType:AVMediaTypeVideo].firstObject;
AVMutableVideoCompositionLayerInstruction *videoLayerInstruction = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:videoTrack];

// Watermark layer above video layer here.
instruction.layerInstructions = @[
  watermarkLayerInstruction,
  videoLayerInstruction,
];
  
videoComposition.instructions = @[instruction];

// Export the video w/ watermark.
AVAssetExportSession *exportSession = ...;
...
exportSession.videoComposition = videoComposition;

...
  

---

And btw, if you just need to add an image as watermark, another solution by using AVVideoComposition's

-videoCompositionWithAsset:applyingCIFiltersWithHandler:

also works well on both real device & simulator, but I tested it and found it's slower. Seems this way is more suitable for video blender/filter.

Seavir

There's a similar question on Apple Developer Forums "Crash When Exporting Video with Text Overlay", answered there as well. – Seavir

I couldn't get this code to work in the simulator either. Can you please post the complete code without removed sections? – Initiation

@NS how about your real device testing version (please make it works first), the commented out code snippets are same. Just need to replace the method I mentioned above based on your version, and provide extra setup like code sample I pasted above. Others are same. – Seavir

@Seavir I got this to work, but the CALayer added to the animation tool is being rendered as opaque (black background) and not transparent (clear background). Any input on this would be much appreciated. – Euphoria

@Euphoria That's off-topic, I can't tell what's the issue w/o having any code snippet provided. Two guesses: 1. your opaque is set to YES in UIGraphicsBeginImageContextWithOptions; 2. you used UIImageJPEGRepresentation instead UIImagePNGRepresentation to generate image data. – Seavir

@Seavir managed to fix the issue by setting the pixel buffer attributes kCVPixelBufferPixelFormatTypeKey to kCVPixelFormatType_32BGRA instead of kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange. – Euphoria

This sounds very promising. Let me undust my old hobby project and try it... – Fred

@NicolasMiari it solved my case at least ;). Btw, if you prefer Swift, someone had provided a Swift version based on my solution in Apple Developer Forums: "Crash When Exporting Video with Text Overlay" (I haven't tested that version, u can just have a try) – Seavir

@Seavir It's not so much an issue of translating your code to Swift, as it is fitting it with my own (and I'm rusty with the whole AVFoundation API), but I'll check that too; thanks! – Fred

Interestingly enough, Apple's programming guide uses the same approach as my original code: developer.apple.com/library/archive/documentation/AudioVideo/… – Fred

@NicolasMiari yeah, about this issue, I've searched a lot including the official doc at that time (and also reached your question here), none covers this case. Even an Apple's engineer said "The crash looks to be fairly deep in the CoreAnimation rendering code. I recommend filing a bug using the Feedback Assistant." Someone mentioned it worked under 15.7 (I hadn't tried), but comes back again under iOS 16. – Seavir

Answer 2

I'm having the same issue. Started after iOS 13.4 and is only shown on the simulator (device is working fine). If I comment out parentLayer.addSublayer(videoLayer) then the app doesn't crash, but the exported video isn't the desired output.

Rorqual

My app crashes on Simulator and Device. – Fred

I too can avert the crash if I comment out that exact same line (remove the video sublayer that works in coordination with the text layer), but the output video results in a black screen. – Fred

Is your device also running 13.4 (or later), and avoiding the issue? – Fred

yea device on 13.4 and running fine. Simulator still crashes on Xcode 11.5 – Rorqual

Still happens on Xcode 12 and iOS 14.0 Simulator :( – Subjection

Any update on this? Still crashing Xcode 12.3, iOS 14.3 :(( – Tallis

Answer 3

This fixed it for me in iOS 14.5:

public static var isSimulator: Bool {
  #if targetEnvironment(simulator)
  true
  #else
  false
  #endif
}

// ...

let export = AVAssetExportSession(
  asset: composition,
  presetName: isSimulator ? AVAssetExportPresetPassthrough : AVAssetExportPresetHighestQuality
)

edit: Doesn't actually render like on a real device though. Edits are simply ignored...

Mithras

Thanks, @Mithras -- This doesn't fix the issue, it just bypasses it when running on the simulator, which is a good start, since the crash seems to only happen on the simulator. With this, at least it doesn't crash and interrupt development. – Paganism

Yes, I noticed that after posting. Changed my answer now – Mithras

Answer 4

#if targetEnvironment(simulator)
    // Adding layers while export crashes on simulator as it expects opaque background.
#else
if let animationTool = getAnimationTool() {
    videoComposition.animationTool = animationTool
}
#endif

Here getAnimationTool() will return AVVideoCompositionCoreAnimationTool. It can be either Image layer or text layer. But should return AVVideoCompositionCoreAnimationTool.

Cherice