CIDetector and UIImagePickerController

Question

I'm trying to implement the built-in iOS 5 face detection API. I'm using an instance of UIImagePickerController to allow the user to take a photo and then I'm trying to use CIDetector to detect facial features. Unfortunately, featuresInImage always returns an array of size 0.

Here's the code:

- (void)imagePickerController:(UIImagePickerController *)picker didFinishPickingMediaWithInfo:(NSDictionary *)info
{
UIImage* picture = [info objectForKey:UIImagePickerControllerOriginalImage];

NSNumber *orientation = [NSNumber numberWithInt:
                         [picture imageOrientation]];
NSDictionary *imageOptions =
[NSDictionary dictionaryWithObject:orientation
                            forKey:CIDetectorImageOrientation];

CIImage *ciimage = [CIImage imageWithCGImage:[picture CGImage]
                                     options:imageOptions];


NSDictionary *detectorOptions =
[NSDictionary dictionaryWithObject:CIDetectorAccuracyLow
                            forKey:CIDetectorAccuracy];
CIDetector *detector = [CIDetector detectorOfType:CIDetectorTypeFace
                                          context:nil
                                          options:detectorOptions];

NSArray *features = [detector featuresInImage:ciimage];
NSLog(@"Feature size: %d", features.count);
}

This always returns 0 features. However, if I use a UIImage from a file built-in to the application, the face detection works great.

I'm using code from this Pragmatic Bookshelf article.

For what it's worth, I think the error is when I convert the UIImage from the camera to a CIImage, but it could be anything.

Inevitable

Answer 1

@tonyc Your solution only works for a specific case that is "UIImageOrientationRight". The problem comes from the difference between the orientation in UIImage and CIDetectorImageOrientation. From iOS docs:

CIDetectorImageOrientation

A key used to specify the display orientation of the image whose features you want to detect. This key is an NSNumber object with the same value as defined by the TIFF and EXIF specifications; values can range from 1 through 8. The value specifies where the origin (0,0) of the image is located. If not present, the default value is 1, which means the origin of the image is top, left. For details on the image origin specified by each value, see kCGImagePropertyOrientation.

Available in iOS 5.0 and later.

Declared in CIDetector.h.

So the problem now is the conversion between these two orientations, here is what I did in my code, I tested and it worked for all orientations:

int exifOrientation;
switch (self.image.imageOrientation) {
    case UIImageOrientationUp:
        exifOrientation = 1;
        break;
    case UIImageOrientationDown:
        exifOrientation = 3;
        break;
    case UIImageOrientationLeft:
        exifOrientation = 8;
        break;
    case UIImageOrientationRight:
        exifOrientation = 6;
        break;
    case UIImageOrientationUpMirrored:
        exifOrientation = 2;
        break;
    case UIImageOrientationDownMirrored:
        exifOrientation = 4;
        break;
    case UIImageOrientationLeftMirrored:
        exifOrientation = 5;
        break;
    case UIImageOrientationRightMirrored:
        exifOrientation = 7;
        break;
    default:
        break;
}

NSDictionary *detectorOptions = @{ CIDetectorAccuracy : CIDetectorAccuracyHigh }; // TODO: read doc for more tuneups
CIDetector *faceDetector = [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:detectorOptions];

NSArray *features = [faceDetector featuresInImage:[CIImage imageWithCGImage:self.image.CGImage]
                                          options:@{CIDetectorImageOrientation:[NSNumber numberWithInt:exifOrientation]}];

Quincyquindecagon

what is self.image.imageOrientation can you please explain? – Draconic

Please advise here #54399574 – Patin

Answer 2

Sure enough, after spending a day looking into this and being stumped, I've found a solution an hour after posting this.

I eventually noticed that the face detection did work in landscape, but not in portrait.

Turns out I needed these options:

NSDictionary *imageOptions = [NSDictionary dictionaryWithObject:[NSNumber numberWithInt:6]
                                                         forKey:CIDetectorImageOrientation];
NSArray *features = [detector featuresInImage:ciimage options:imageOptions];

Inevitable

Swift4.2 var imageOptions = [CIDetectorImageOrientation : NSNumber(value: 6)] var features = faceDetector!.features(in: myImage, options: imageOptions as? [String : Any]) – Judejudea

Answer 3

Swift 3 Version

var exifOrientation : Int
switch (inputImage.imageOrientation)
{
case UIImageOrientation.up:
    exifOrientation = 1;
case UIImageOrientation.down:
    exifOrientation = 3;
case UIImageOrientation.left:
    exifOrientation = 8;
case UIImageOrientation.right:
    exifOrientation = 6;
case UIImageOrientation.upMirrored:
    exifOrientation = 2;
case UIImageOrientation.downMirrored:
    exifOrientation = 4;
case UIImageOrientation.leftMirrored:
    exifOrientation = 5;
case UIImageOrientation.rightMirrored:
    exifOrientation = 7;
}

let ciImage = CIImage(cgImage: inputImage.cgImage!)
let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
let faces = faceDetector.features(in: ciImage, options:["CIDetectorImageOrientation":exifOrientation])

Kreit

Answer 4

I had a similar problem - the face detector was working in orientations that didn't match the image orientation property. iOS face detector orientation and setting of CIImage orientation

a quick work around that I found to help with that is to disregard the image orientation all together and "flatten" the image (so that there is no image orientation data)

I used this code:

http://blog.logichigh.com/2008/06/05/uiimage-fix/

and sure enough looks like it is working much better for front camera pics. I haven't tried enough on landscape to tell if it has helped there but looks promising

Mascle

Your link is broken – Judejudea