Is there a way to read data from CGImage without internal caching?

Question

I am fighting with an internal caching (about 90 MB for 15 mp image ) in CGContextDrawImage/CGDataProviderCopyData functions.
Here is the stack-trace in profiler:

In all cases, IOSurface is created as a "cache", and isn't cleaned after @autoreleasepool is drained.
This leaves a very few chances for an app to survive.
Caching doesn't depend on image size: I tried to render 512x512, as well as 4500x512 and 4500x2500 (full-size) image chunks.

I use @autoreleasepool, CFGetRetainCount returns 1 for all CG-objects before cleaning them.

The code which manipulates the data:

+ (void)render11:(CIImage*)ciImage fromRect:(CGRect)roi toBitmap:(unsigned char*)bitmap {
    @autoreleasepool
    {
        int w = CGRectGetWidth(roi), h = CGRectGetHeight(roi);

        CIContext* ciContext = [CIContext contextWithOptions:nil];
        CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();

        CGContextRef cgContext = CGBitmapContextCreate(bitmap, w, h,
                                                   8, w*4, colorSpace,
                                                   kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);


        CGImageRef cgImage = [ciContext createCGImage:ciImage
                                         fromRect:roi
                                           format:kCIFormatRGBA8
                                       colorSpace:colorSpace
                                         deferred:YES];


        CGContextDrawImage(cgContext, CGRectMake(0, 0, w, h), cgImage);

        assert( CFGetRetainCount(cgImage) == 1 );

        CGColorSpaceRelease(colorSpace);
        CGContextRelease(cgContext);
        CGImageRelease(cgImage);
    }
}

What I know about IOSurface: it's from the previously private framework IOSurface.
CIContext has a function render: ... toIOSurface:.
I've created my IOSurfaceRef and passed it to this function, and the internal implementation still creates its own surface, and doesn't clean it.

So, do you know (or assume):
1. Are there other ways to read CGImage's data buffer except CGContextDrawImage/CGDataProviderCopyData ?
2. Is there a way to disable caching at render?
3. Why does the caching happen?
4. Can I use some lower-level (while non-private) API to manually clean up system memory?

Any suggestions are welcome.

Setup

Two thoughts: Have you tried calling clearCaches on the ciContext after you created the cgImage? Also you could try to init the ciContext with contextWithCGContext:options: after creating the cgContext, passing it as an argument. That should tell Core Image to render directly into that context (instead of an intermediate buffer) and you don't need to call CGContextDrawImage. I haven't tried it, though. – Lacker

@FrankSchlegel Thanks! 1) Yes, I tried, it does nothing. 2) No (only [CIContext contextWithOptions:nil]), I'll try. – Setup

Any success? I'm curious if this worked. – Lacker

@FrankSchlegel Well, based on your advice I tried [CIContext contextWithOptions:] and passed kCIContextCacheIntermediates:NO. The function CreateCachedSurface is still called inside [CIContext render: toBitmap:]. – Setup

CoreImage caching happens because Lanczos requires intermediate because it is implemented as a vertical and horizontal passes. These surfaces will remain after the render is complete but they are marked as volatile so they will have no impact on jetsam pressure. Also note that [cictx clearCaches] will empty (but not delete) the surfaces associated with the cictx instance. – Guggenheim

Answer 1

To answer your second question,

Is there a way to disable caching at render?

setting the environment variable CI_SURFACE_CACHE_CAPACITY to 0 will more-or-less disable the CIContext surface cache. Moreover, you can specify a custom (approximate) cache limit by setting that variable to a given value in bytes. For example, setting CI_SURFACE_CACHE_CAPACITY to 2147483648 specifies a 2 GiB surface cache limit.

Note it appears that all of a process's CIContext instances share a single surface cache. It does not appear to be possible to use separate caches per CIContext.

Leaper

Thanks, your advice is fantastic!!! I set CI_SURFACE_CACHE_CAPACITY=0, and that "more-or-less" is 200 KB cache instead of 90 MB. While I'm still testing for other photos. Do you know why isn't this variable documented? Google search returns 2 hits for "CI_SURFACE_CACHE_CAPACITY". – Setup

Apple probably considers surface caching in CIContext an implementation detail subject to change at any time. They don't want to document something unless they are going to support it long term. That's my speculation. Don't be surprised if this fix stops working in a future version of macOS. Hopefully if it does stop working there will be another solution to this problem, or the problem will go away altogether. – Leaper

Answer 2

If you just need to manipulate CIImage data, may consider to use CIImageProcessorKernel to put data into CPU or GPU calculation without extracting them.

I notice that

[ciContext render:image toBitmap:bitmap rowBytes: w*4 bounds:image.extent format:kCIFormatRGBA8 colorSpace:colorSpace];

There is no such 90M cache. Maybe it's what you want.

Sorehead

Thanks! I do need to read the data into unsigned char* buffer. Is it possible with CIImageProcessorKernel ? – Setup

It depends on knowledge level. Baseaddress has the data you need. If you cannot get it out, Simply way is to render to somewhere like iosurface or screen then capture screen. – Sorehead

cgimage can render directly to data. If i call createCGImage, it's 260M. call this it's only 170M during peak. – Sorehead

"Baseaddress has the data you need" - thanks, I'll try. "cgimage can render directly to data. If i call createCGImage" - yeah, but CGContextDrawImage does that unwanted caching inside – Setup

sorry, not CGImage but CIImage. In my answer, I upgrade with CIcontext to render a CIImage to data without cgimage. – Sorehead

Sorry, [ciContext render:image toBitmap:bitmap - this was the intial code which does has a leak. All CIContext render* methods have it. – Setup

Could you clarify: "Baseaddress has the data you need" - is it in CIImageProcessorKernel approach? – Setup

"Baseaddress has the data you need." Yes, but you may not get it out of the block. – Sorehead

I did not see get leaked portion. [ciContext render:image toBitmap:bitmap]. – Sorehead