Having trouble creating UIImage from CIImage in iOS5

Question

I'm using the AVFoundation framework. In my sample buffer delegate I have the following code:

-(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection{
     CVPixelBufferRef pb = CMSampleBufferGetImageBuffer(sampleBuffer);
     CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pb];
     self.imageView.image = [UIImage imageWithCIImage:ciImage];
}

I am able to use the CIImage to run the face detector etc. but it does not show up in the UIImageView ... the imageView remains white. Any ideas as to the problem? I am using the following to setup my session:

    self.session = [[AVCaptureSession alloc] init];
self.session.sessionPreset = AVCaptureSessionPreset640x480;
self.videoDevice = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
self.videoInput = [AVCaptureDeviceInput deviceInputWithDevice:videoDevice error:nil]; 
self.frameOutput = [[AVCaptureVideoDataOutput alloc] init];
self.frameOutput.videoSettings = [NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey];

Dachia

Answer 1

This might help. I was having the same issue (image not being drawn to screen) with this code:

CIImage *image = [[CIImage alloc] initWithImage:[UIImage imageNamed:@"mushroom.jpg"]];

theImageView.image = [UIImage imageWithCIImage:image];

However, after changing the code to this, it now works correctly:

CIImage *image = [[CIImage alloc] initWithImage:[UIImage imageNamed:@"mushroom.jpg"]];

CIContext *context = [CIContext contextWithOptions:nil];

theImageView.image = [UIImage imageWithCGImage:[context createCGImage:image fromRect:image.extent]];

To read more about CIContext take a look here: http://developer.apple.com/library/ios/#DOCUMENTATION/GraphicsImaging/Reference/QuartzCoreFramework/Classes/CIContext_Class/Reference/Reference.html#//apple_ref/occ/cl/CIContext

Hydrocortisone

Thanks -- this does indeed let me draw the image. However it leaks memory causing the app to crash after running a few seconds. – Dachia

Yea, I haven't really looked into it too much yet. I'm sure you have to release the contexts or the CGImageRef at some point. Also I'm sure the method above isn't the practical way of using CIImage views. I plan to watch the "Using Core Image on iOS & Mac OS X" video (which is on the Apple Developer site, WWDC 2011 Session Videos) tomorrow and learn more in-depth, I'll drop a line if I learn anything useful. – Hydrocortisone

I recommend you also watch this video and/or look at the presentation slides as it explains how to use Core Image, and goes into detail about contexts and when to use CPU/GPU and more – Hydrocortisone

I also found this to be useful: OpenGL ES Programming Guide for iOS developer.apple.com/library/ios/#documentation/3DDrawing/… – Hydrocortisone

Would you mind explaining why this works vs. imageWithCIImage? I agree with you, I just can't understand why imageWithCIImage alone does not work, as it should according to the docs. – Sunless

It sounds like imageWithCGImage does not retain the given CGImage. When you call _create_CGImage you get a non-autoreleased object, so it is not freed - This is probably the cause for it working but also leaking... – District

@Sunless here's what UIImage reference says about its CGImage property: "If the UIImage object was initialized using a CIImage object, the value of the property is NULL". and it looks like UIImageView tries to access UIImage's CGImage in its implementation. this also looks to be the case for UIImagePNGRepresentation function - it returns nil if UIImage is not CGImage-based. But +[CIContext createCGImage:] just does what it says - creates CGImage, so it works – Glia

A good answer by Tulca, @Sunless you should select this answer. – Bronder

@C4-Travis it's not my question to answer. – Sunless

@Sunless oops! sorry for that. – Bronder

@Glia thanks for finding the explanation! No way I would have figured it out on my own. Also, I will point out that we have to use alloc/init instead of the convenience methods for creating the CIImage to get this to work. – Richmound

@Hydrocortisone Your code leaks: [context createCGImage...] creates a CGContextRef that must be released (even in ARC). Since you don't assign it to a variable, you cannot release it, and you leak quite a bit of memory here. – Chil

Answer 2

Here is one way that I got it to work:

CIContext *context = [CIContext contextWithOptions:nil];
CGImageRef ref = [context createCGImage:result fromRect:ciImage.extent];
self.imgView.image = [UIImage imageWithCGImage:ref scale:1.0 orientation:UIImageOrientationRight];
CGImageRelease(ref);

Note: Creating the context multiple times is really bad and actually causes a memory leak. You should just create the context once as a property in your class instance and reuse it!

Dachia

Answer 3

CIImage *ciImage = [UIImage imageNamed:@"imageName.png"].CIImage;
UIImage *uiImage = [[UIImage alloc] initWithCIImage:ciImage];

Torrell

Did not work for me. I created the CIImage in a different way than shown here. – Pretzel

Answer 4

Here is a complete code sample that I've used in my projects.

- (UIImage *)makeUIImageFromCIImage:(CIImage *)ciImage {
    CIContext *context = [CIContext contextWithOptions:nil];
    CGImageRef cgImage = [context createCGImage:ciImage fromRect:[ciImage extent]];

    UIImage* uiImage = [UIImage imageWithCGImage:cgImage];
    CGImageRelease(cgImage);

    return uiImage;
}

CIImage is basically a recipe for an image. You will run into problems if you try to directly convert from CIImage to UIImage. For example, calling

NSData * UIImageJPEGRepresentation (
   UIImage *image,
   CGFloat compressionQuality
);

will return nil.

Centring

brilliant, thanks!!! I wonder why Apple provides [UIImage imageWithCIImage] if it rarely works. Thanks Apple. – Idiocy

I ran into some very subtle color distortion and alpha blending issues with this method, fixed by setting color space context = [CIContext contextWithOptions:@{kCIContextWorkingColorSpace : [NSNull null]}]; – Mella

Thanks @Mella that was the critical piece of information we needed – Mcgaw