Swift 4, Subclassing CIFilter crashes only with "input" instance variables

Question

How do you subclass CIFilter now? In Swift 3 I could do this as a simple example:

class CustomFilter: CIFilter {
   var inputImage: CIImage?
   var inputOrigin: CIVector?
   var inputAnotherVar: String?
}

But in Swift 4 I get an NSException. If I remove "input" from each variable it works fine. I could just do that. But I feel like I'm missing something important and I can't seem to find anything explaining this behaviour.

This compiles fine in Swift4:

class CustomFilter: CIFilter {
   var image: CIImage?
   var origin: CIVector?
   var anotherVar: String?
}

Here's the error in a Playground:

Cristophercristy

It appears (to me) like you are missing some code. Why exactly is inputImage a CIVector and not some kind of image (probably CIImage)? And if that's not the issue, maybe you could provide more code about what makes CustomFilter? – Geosphere

Thanks for taking a look! You're right the name was misleading. But I've re-edited the question to hopefully make it more clear what I mean. – Cristophercristy

I may be a bit confused still, but I use a CIFilter or CIKernel three ways - only one of which requires subclassing CIFilter. I checked my code (both Swift 3 and 4) and - for instance - public var inputImage: CIImage? both builds and works just fine for me. Beyond declaring my class public (it's part of a framework target) I'm not seeing why you are having an issue. Are you getting a build error? A runtime error? – Geosphere

Really, that's bizarre for me then. It happens at runtime. I've attached a photo of the error from a playground. – Cristophercristy

I'll post some code as an answer. I'm not seeing where it will address your issue, but maybe something will jump out at you. – Geosphere

Code posted. One more note: I haven't tried things in a playground. Have you tried this code in an app project? (And yet one last note: I've only used this code in iOS, not macOS.) – Geosphere

Thanks for that! I had only tried to run it in a playground. But it works in a app project. Oddly been using Simon's book playing around with examples and then all of a sudden this error popped up when i updated to Swift4. Still not sure why, but it's at least it's just contained to a Playground for now. – Cristophercristy

I still couldn't get custom CIFilter to work with Swift 4. Even when the custom filter does nothing by setting outputImage to be inputImage. The codes crashed at filter.setValue(someInputCIImage, forKey: kCIInputImageKey). Any idea? – Squishy

Have you tried in an app? And not just a playground. For some reason when I updated Xcode my code would work in an app project. But not in a playground. – Cristophercristy

Answer 1

Based on the comments, here's some Swift 4 code (unchanged from Swift 3) that both builds and executes. I'm not seeing where your issue is, so if this doesn't help you out, comment on it and I'll delete it. (If it does help, I'll edit my answer to be more specific!)

The first CIFilter uses CIColorInvert and CIHeightFieldFromMask to create a "text mask" based on the text in a UILabel. It also overrides the outputImage property of CIFilter. The second CIFilter is actually a "wrapper" around a CIKernel, using an CIImage as inputImage, the mask (from the first filter) as inputMask and also overrides outputImage like the first does.

Virtually all of this code was lifted from Core Image for Swift by Simon Gladman, nowadays available as an iBook for free. While the book was written in Swift 2, I find it to be a valuable resource for working with Core Image.

(Side note: The book combines all of this. I split out things as I was working on adapting this into an existing app as a watermark. I ended up going a different route!)

Mask.swift

public class Mask: CIFilter {
    public var inputExtent:CGRect?
    var inputRadius: Float = 15 {
        didSet {
            if oldValue != inputRadius {
                refractingImage = nil
            }
        }
    }
    private var refractingImage: CIImage?
    private var rawTextImage: CIImage?

    override public var outputImage: CIImage! {
        if refractingImage == nil {
            generateRefractingImage()
        }
        let mask = refractingImage?.applyingFilter("CIColorInvert", parameters: [:])
        return mask
    }

    func generateRefractingImage() {
        let label = UILabel(frame: inputExtent!)
        label.text = "grand canyon"
        label.font = UIFont.boldSystemFont(ofSize: 300)
        label.adjustsFontSizeToFitWidth = true
        label.textColor = UIColor.white

        UIGraphicsBeginImageContextWithOptions(
            CGSize(width: label.frame.width,
                   height: label.frame.height), true, 1)
        label.layer.render(in: UIGraphicsGetCurrentContext()!)
        let textImage = UIGraphicsGetImageFromCurrentImageContext()
        UIGraphicsEndImageContext()

        rawTextImage = CIImage(image: textImage!)!
        refractingImage = CIFilter(name: "CIHeightFieldFromMask",
                                   withInputParameters: [
                                    kCIInputRadiusKey: inputRadius,
                                    kCIInputImageKey: rawTextImage!])?.outputImage?
            .cropped(to: inputExtent!)
    }
}

Refraction.swift

public class Refraction: CIFilter {
    public var inputImage: CIImage?
    public var inputMask:CIImage?

    var inputRefractiveIndex: Float = 4.0
    var inputLensScale: Float = 50
    public var inputLightingAmount: Float = 1.5

    var inputLensBlur: CGFloat = 0
    public var inputBackgroundBlur: CGFloat = 2

    var inputRadius: Float = 15

    override public func setDefaults()
    {
        inputRefractiveIndex = 4.0
        inputLensScale = 50
        inputLightingAmount = 1.5
        inputRadius = 15
        inputLensBlur = 0
        inputBackgroundBlur = 2
    }

    override public var outputImage: CIImage! {
        guard let inputImage = inputImage, let refractingKernel = refractingKernel else {
            return nil
        }

        let extent = inputImage.extent
        let arguments = [inputImage,
                         inputMask!,
                         inputRefractiveIndex,
                         inputLensScale,
                         inputLightingAmount] as [Any]
        return refractingKernel.apply(extent: extent,
                                      roiCallback: {
                                        (index, rect) in
                                        return rect
        },
                                      arguments: arguments)!
    }

    let refractingKernel = CIKernel(source:
        "float lumaAtOffset(sampler source, vec2 origin, vec2 offset)" +
            "{" +
            " vec3 pixel = sample(source, samplerTransform(source, origin + offset)).rgb;" +
            " float luma = dot(pixel, vec3(0.2126, 0.7152, 0.0722));" +
            " return luma;" +
            "}" +


            "kernel vec4 lumaBasedRefract(sampler image, sampler refractingImage, float refractiveIndex, float lensScale, float lightingAmount) \n" +
            "{ " +
            " vec2 d = destCoord();" +

            " float northLuma = lumaAtOffset(refractingImage, d, vec2(0.0, -1.0));" +
            " float southLuma = lumaAtOffset(refractingImage, d, vec2(0.0, 1.0));" +
            " float westLuma = lumaAtOffset(refractingImage, d, vec2(-1.0, 0.0));" +
            " float eastLuma = lumaAtOffset(refractingImage, d, vec2(1.0, 0.0));" +

            " vec3 lensNormal = normalize(vec3((eastLuma - westLuma), (southLuma - northLuma), 1.0));" +

            " vec3 refractVector = refract(vec3(0.0, 0.0, 1.0), lensNormal, refractiveIndex) * lensScale; " +

            " vec3 outputPixel = sample(image, samplerTransform(image, d + refractVector.xy)).rgb;" +

            " outputPixel += (northLuma - southLuma) * lightingAmount ;" +
            " outputPixel += (eastLuma - westLuma) * lightingAmount ;" +

            " return vec4(outputPixel, 1.0);" +
        "}"
    )
}

Usage

let filterMask = Mask()
let filter = Refraction()
var imgOriginal:CIImage!
var imgMask:CIImage!
var imgEdited:CIImage!

// I have a set of sliders that update a tuple and send an action that executes the following code

filterMask.inputRadius = sliders.valuePCP.3
imgMask = filterMask.outputImage
filter.inputMask = imgMask
filter.inputRefractiveIndex = sliders.valuePCP.0
filter.inputLensScale = sliders.valuePCP.1
filter.inputLightingAmount = sliders.valuePCP.2
imgEdited = filter.outputImage

Hope this helps!

Geosphere

Answer 2

I ran into this issue (same "error: Execution was interrupted, reason: EXC_BAD_INSTRUCTION...") in Swift 4 while experimenting with Simon Gladman's "Core Image for Swift." I also tried running example code in an app instead of a playground. The solution for me was to add an @objc dynamic in front of var inputImage: CIImage? In your code it would look like this:

class CustomFilter: CIFilter {
    @objc dynamic var inputImage: CIImage?
    var inputOrigin: CIVector?
    var inputAnotherVar: String?
}

As I understand it, this is because Swift 4 by default minimizes inference so as to reduce binary code size. In contrast, Swift 3 implicitly infers Objc attributes. What this means in practice, is I have to add the @objc dynamic to certain variables that will take advantage of Objective-C's dynamic dispatch, such as when setting a CoreImage filter: filter.setValue(inputImage, forKey: kCIInputImageKey). Here are some resources that describe similar issues with Swift's static dispatch using Swift4 when dealing with Obj-C API's that rely on dynamic dispatch and how to deal with dispatch when you migrate from Swift 3 to 4.

Aphrodisiac

I had to put @objc dynamic in front of every variable for it to compile in a playground. In an app I didn't even need to use @objc dynamic....weird. – Cristophercristy