Should I set a color space for a CIImage or a CIContext?

Question

Core Image lets us specify a color space for a CIContext, as in:

let context = CIContext(options: [kCIContextOutputColorSpace: NSNull(),
                        kCIContextWorkingColorSpace: NSNull()])

Or for a CIImage, as in:

let image = CIImage(cvImageBuffer: inputPixelBuffer,
                    options: [kCIImageColorSpace: NSNull()])

How are these three related:

• kCIContextOutputColorSpace
• kCIContextWorkingColorSpace
• kCIImageColorSpace

What are the pros and cons of setting each of them?

Screw

I'm not able to give you any "definitive" answer, but I'll guess it may well depend on your needs. Using a CIContext is costly in terms of performance, so if you can stick with one - required if you are using a GLKView - then do that (stick with one that is). A CIImage is not an image, instead it's a "recipe" to draw or manipulate an image. None of this explains which is best for your needs though, so I guess the question for you is why are you setting a color space? It's that answer that will probably dictate which place to set things. – Technetium

I'm trying to follow Apple's recommendation to optimise my app by not doing color space conversions: developer.apple.com/library/content/documentation/… But when I set kCIImageColorSpace to nil, the gamma is messed up, and I have to set one or more of the other two keys listed in the question. At that point, I no longer knew what I was doing :) So I wanted to understand. Thanks for your help. – Screw

Answer 1

Apple's documentation explains the differences:

workingColorSpace

A key for the color space to use for image operations.

The working color space determines the color space used when executing filter kernels; Core Image automatically converts to and from the source and destination color spaces of input images and output contexts. You specify a working color space using the workingColorSpace key in the options dictionary when creating a Core Image context.

outputColorSpace

A key for the color space to use for images before they are rendered to the context. By default, Core Image uses the GenericRGB color space, which leaves color matching to the system.

To request that Core Image perform no color management, specify the NSNull object as the value for this key. Use this option for images that don’t contain color data (such as elevation maps, normal vector maps, and sampled function tables).

If images are tagged with a color space, they are converted to linear working space before filtering. If you tag a CIImage with DeviceRGB, it is gamma corrected to linear before filtering.

Setting the keys to NSNull instructs CoreImage to leave the color values as they are which is referred to as unmanaged color space.

Scrimshaw

Answer 2

We can classify color spaces into at least three categories. When you initialize a CIImage from a bitmap or data, there is an initializing color space. When you do filter operations, there is a working color space. When you render a CIImage to a destination, there is an output color space.

Initializing Color Spaces

• Bitmap

When you initialize a CIImage from a bitmap, the initializer init(bitmapData data: Data, bytesPerRow: Int, size: CGSize, format: CIFormat, colorSpace: CGColorSpace?) has a color space parameter, which is relatively easy to understand.

• CVPixelBuffer or other formats

When you initialize a CIImage from a CVPixelBuffer, you can optionally provide a color space through the CIImageOption.colorSpace option key. But in most cases, a CVPixelBuffer has already color space information in it, so this is not necessary. And if you provide a different color space, you can get the wrong colors.

However it is acceptable to provide an NSNull() like CIImage(cvPixelBuffer: buffer, options: [CIImageOption.colorSpace : NSNull()]). This means discarding the color space information. If you do this, you typically should set the workingColorSpace to NSNull() too, otherwise the color space of the image will be considered as the workingColorSpace when you apply filters, see below. You typically want the color space back when you render it to a destination, you don't want to lose it permanently, see the "Output Color Spaces" section below.

Working Color Spaces

CIContext has a property workingColorSpace. The document says

The working color space determines the color space used when executing filter kernels; Core Image automatically converts to and from the source and destination color spaces of input images and output contexts...

This color space is important and can have a visible impact when you use filters such as CIFilter.colorMatrix to transform colors.

You can consider that the context first converts the input image into the working color space, performs filter calculations in the working color space, and then converts it into the output color space.

Note, after applying filters, the colorSpace property of the CIImage may become nil. The system is informing you this way that the color space is not determined yet.

The workingColorSpace is a read-only property, you set it in the initializer like CIContext(options: [CIContextOption.workingColorSpace : working])

If you don't provide a working color space in the initializer, the context will still have a default color space which is linearSRGB according to my test. You can also provide an NSNull() which means discarding the color space information in the input image and no color conversion. This is useful for example with statistical filters such as CIFilter.areaHistogram or when you want to avoid color conversion overhead. If it is the later case, remember to set the output color space back.

Output Color Spaces

A CIImage is just an image "recipe" that contains information on how to produce an image. Ultimately you will need to render it into a destination such as a bitmap or a CGImage. That is when you specify the output color space. This kind of color space may not always have a visible impact when the output is a CGImage and the color space is not nil (I do not recommend creating a CGImage with a nil output color space, the meaning is not clear), since it is just the internal encoding of the data, unless there are extreme colors that cannot be accurately encoded by the color space.

• Bitmap

You can specify the output color space in the render method of CIContext.

• CGImage

You can specify the output color space in the createCGImage(_: CIImage, from: CGRect, format: CIFormat, colorSpace: CGColorSpace?) method of CIContext.

The CIContextOption.outputColorSpace is intended to be used when you call the simpler createCGImage(_: CIImage, from: CGRect) method overload. If you do not provide this key, the system will choose DeviceRGB according to my test, in this case the color may not be consistent across devices. If you call the above complex overload createCGImage(_: CIImage, from: CGRect, format: CIFormat, colorSpace: CGColorSpace?), this option will be ignored. I suggest using the complex overload since it is more explicit.

Skimmer