iPhone image orientation wrong when resizing with SkiaSharp

Question

I'm using the SKBitmap.Resize() method in SkiaSharp on a Xamarin.Forms project to resize images for display. The problem I'm encountering is when taking a photo on iOS, when a photo is taken in portrait, the image is displayed with the right side up. Taking a photo on Android, importing from the photo gallery on both an Android and iOS device maintains orientation, but taking a photo in iOS does not. If I don't resize the image using SkiaSharp (just display the image without any resizing), then the image displays with the proper orientation. However that is not a solution as the images need to be resized. Below is my code -

private byte[] GetResizedImageData(string imageName)
    {
        float resizeFactor = 0.5f;
        var filePath = PathUtil.GetImagePath(imageName);
        var ogBitmap = SKBitmap.Decode(filePath);

        float fWidth = ogBitmap.Width * resizeFactor;
        int width = (int) Math.Round(fWidth);

        float fHeight = ogBitmap.Height * resizeFactor;
        int height = (int) Math.Round(fHeight);

        if (height >= 4096 || width >= 4096)
        {
            width = width * (int)resizeFactor;
            height = height * (int)resizeFactor;
        }

        var scaledBitmap = ogBitmap.Resize(new SKImageInfo( width, height), SKBitmapResizeMethod.Box);
        var image = SKImage.FromBitmap(scaledBitmap);
        var data = image.Encode(SKEncodedImageFormat.Jpeg, 100);

        return data.ToArray();
    }

PathUtil.GetImagePath() is just a helper to get platform-specific paths for where the photos are being stored.

Skolnik

You should always read the native orientation on the image (EXIF) as iPhones portraits are usually tagged as UIImageOrientation.Right, On Android things get really messy as different manufactures sometimes mount their camera sensors 90 degrees clockwise or counterclockwise (Samsumg and LG are notorious for 90 rotation, I even used a couple of Chinese Android devices where the sensors are installed 180 degrees, most of the time this is due to physical manufacturing and packaging constraints, moral of the story, read the image rotation before applying any transformations... ;-) – Unrig

You can obtain the Exif orientation via SKCodec.Origin and from the SKCodecOrigin determine the appropriate transformation that you need to apply. – Unrig

So after grabbing that, I can't seem to change the SKCodec.Origin as it's read only and I'm outputting the image as a byte array to Xamarin.Forms Image.ImageSource where I can't seem to handle the rotation/transformation either. I think I'll have to add in platform specific code to handle the bitmap orientation change. – Skolnik

Answer 1

SkiaSharp didn't actually provide a method for me to manipulate and change the orientation of the image. In the end I ended up altering the orientation as I captured and saved the image using platform specific code.

Skolnik

Answer 2

For those with the same issue I did the following and would gladly accept input on improvements.

        public static SKBitmap HandleOrientation(SKBitmap bitmap, SKCodecOrigin orientation)
    {
        SKBitmap rotated;
        switch (orientation)
        {
            case SKCodecOrigin.BottomRight:

                using (var surface = new SKCanvas(bitmap))
                {
                    surface.RotateDegrees(180, bitmap.Width / 2, bitmap.Height / 2);
                    surface.DrawBitmap(bitmap.Copy(), 0, 0);
                }

                return bitmap;

            case SKCodecOrigin.RightTop:                                                 
                rotated = new SKBitmap(bitmap.Height, bitmap.Width);

                using (var surface = new SKCanvas(rotated))
                {
                    surface.Translate(rotated.Width, 0);
                    surface.RotateDegrees(90);
                    surface.DrawBitmap(bitmap, 0, 0);
                }

                return rotated;

            case SKCodecOrigin.LeftBottom:
                rotated = new SKBitmap(bitmap.Height, bitmap.Width);

                using (var surface = new SKCanvas(rotated))
                {
                    surface.Translate(0, rotated.Height);
                    surface.RotateDegrees(270);
                    surface.DrawBitmap(bitmap, 0, 0);
                }

                return rotated; 

            default:                       
                return bitmap;            
        }

And then use the following to get the original orientation.

            // TODO: Improve this.. I do not know how to "reset" 
            // the inputStream, so new it up twice. :/
            using (var inputStream = new SKManagedStream(imageIn))
            {
                using (var codec = SKCodec.Create(inputStream))
                {
                    orientation = codec.Origin;
                }
            }

....... Then

SKBitmap orientedWExif = HandleOrientation(resized, orientation);

Flautist

upvoted for the information on how to get the origin from the stream (or byte[]): SKCodec.Create, thank you – Lamori

Answer 3

SHORT SOLUTION:

1- Make sure SkiaSharp is updated (above 2.88.0).

2- Use SKImage before SKBitmap like bellow:

using SKImage image = SKImage.FromEncodedData(filePath);

using SKBitmap ogBitmap = SKBitmap.FromImage(image);

Problem solved.

Reive

This is by far the easiest solution now – Perspicuous

works for my Andriod but not iOS – Gibbon

Answer 4

Workable solution for resize and handle orientation

public class ImageResizer : IImageResizer
{
    private const int Quality = 75;

    public byte[] Resize(byte[] data, int newWidth, int newHeight)
    {
        using (var inputStream = new SKMemoryStream(data))
        {
            using (var codec = SKCodec.Create(inputStream))
            {
                using (var original_old = SKBitmap.Decode(codec))
                {
                    int sourceWidth = original_old.Width;
                    int sourceHeight = original_old.Height;

                    float nPercentW = ((float) newWidth / (float) sourceWidth);
                    float nPercentH = ((float) newHeight / (float) sourceHeight);

                    float nPercent = nPercentH < nPercentW ? nPercentH : nPercentW;

                    int destWidth = (int) (sourceWidth * nPercent);
                    int destHeight = (int) (sourceHeight * nPercent);

                    using (SKBitmap original = original_old.Resize(new SKImageInfo(destWidth, destHeight), SKFilterQuality.Medium))
                    {

                        var useWidth = original.Width;
                        var useHeight = original.Height;
                        Action<SKCanvas> transform = canvas => { };
                        switch (codec.EncodedOrigin)
                        {
                            case SKEncodedOrigin.TopLeft:
                                break;
                            case SKEncodedOrigin.TopRight:
                                // flip along the x-axis
                                transform = canvas => canvas.Scale(-1, 1, useWidth / 2, useHeight / 2);
                                break;
                            case SKEncodedOrigin.BottomRight:
                                transform = canvas => canvas.RotateDegrees(180, useWidth / 2, useHeight / 2);
                                break;
                            case SKEncodedOrigin.BottomLeft:
                                // flip along the y-axis
                                transform = canvas => canvas.Scale(1, -1, useWidth / 2, useHeight / 2);
                                break;
                            case SKEncodedOrigin.LeftTop:
                                useWidth = original.Height;
                                useHeight = original.Width;
                                transform = canvas =>
                                {
                                    // Rotate 90
                                    canvas.RotateDegrees(90, useWidth / 2, useHeight / 2);
                                    canvas.Scale(useHeight * 1.0f / useWidth, -useWidth * 1.0f / useHeight, useWidth / 2, useHeight / 2);
                                };
                                break;
                            case SKEncodedOrigin.RightTop:
                                useWidth = original.Height;
                                useHeight = original.Width;
                                transform = canvas =>
                                {
                                    // Rotate 90
                                    canvas.RotateDegrees(90, useWidth / 2, useHeight / 2);
                                    canvas.Scale(useHeight * 1.0f / useWidth, useWidth * 1.0f / useHeight, useWidth / 2, useHeight / 2);
                                };
                                break;
                            case SKEncodedOrigin.RightBottom:
                                useWidth = original.Height;
                                useHeight = original.Width;
                                transform = canvas =>
                                {
                                    // Rotate 90
                                    canvas.RotateDegrees(90, useWidth / 2, useHeight / 2);
                                    canvas.Scale(-useHeight * 1.0f / useWidth, useWidth * 1.0f / useHeight, useWidth / 2, useHeight / 2);
                                };
                                break;
                            case SKEncodedOrigin.LeftBottom:
                                useWidth = original.Height;
                                useHeight = original.Width;
                                transform = canvas =>
                                {
                                    // Rotate 90
                                    canvas.RotateDegrees(90, useWidth / 2, useHeight / 2);
                                    canvas.Scale(-useHeight * 1.0f / useWidth, -useWidth * 1.0f / useHeight, useWidth / 2, useHeight / 2);
                                };
                                break;
                        }

                        var info = new SKImageInfo(useWidth, useHeight);
                        using (var surface = SKSurface.Create(info))
                        {
                            using (var paint = new SKPaint())
                            {
                                // high quality with antialiasing
                                paint.IsAntialias = true;
                                paint.FilterQuality = SKFilterQuality.High;

                                // rotate according to origin
                                transform.Invoke(surface.Canvas);

                                // draw the bitmap to fill the surface
                                surface.Canvas.DrawBitmap(original, info.Rect, paint);
                                surface.Canvas.Flush();

                                using (SKImage image = surface.Snapshot())
                                {
                                    return image.Encode(SKEncodedImageFormat.Jpeg, Quality).ToArray();
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

Governorship

Answer 5

Thank you @ttugates Your answer helped me to fix issue with the orientation rotation. Just want to update your answer as there is some deprecated code.

 using (var inputStream = new SKManagedStream(await file.ReadAllStreamAsync()))
                {
                    using (var codec = SKCodec.Create(inputStream))
                    {
                          orientation = codec.EncodedOrigin;
                    }
                }
  public static SKBitmap HandleOrientation(SKBitmap bitmap, SKEncodedOrigin orientation)
    {
        SKBitmap rotated;
        switch (orientation)
        {
            case SKEncodedOrigin.BottomRight:

                using (var surface = new SKCanvas(bitmap))
                {
                    surface.RotateDegrees(180, bitmap.Width / 2, bitmap.Height / 2);
                    surface.DrawBitmap(bitmap.Copy(), 0, 0);
                }

                return bitmap;

            case SKEncodedOrigin.RightTop:
                rotated = new SKBitmap(bitmap.Height, bitmap.Width);

                using (var surface = new SKCanvas(rotated))
                {
                    surface.Translate(rotated.Width, 0);
                    surface.RotateDegrees(90);
                    surface.DrawBitmap(bitmap, 0, 0);
                }

                return rotated;

            case SKEncodedOrigin.LeftBottom:
                rotated = new SKBitmap(bitmap.Height, bitmap.Width);

                using (var surface = new SKCanvas(rotated))
                {
                    surface.Translate(0, rotated.Height);
                    surface.RotateDegrees(270);
                    surface.DrawBitmap(bitmap, 0, 0);
                }

                return rotated;

            default:
                return bitmap;
        }
    }

Scarce

@AmitaiIrron yes I wanted to comment but didn't have enough reputation points ...it didn't allow me to comment but it allowed to comment on my own answer thank you stackoverflow.com/users/4022580/amitai-irron – Scarce