Android CameraX image rotated

Question

I have followed Google CameraX code lab to implement custom camera. Camera preview is fine but when i take image after image capture image is rotated. I am taking image in portrait mode but saved image is in landscape. Here is the method to configure camera

private fun startCamera() {

    val cameraProviderFuture = ProcessCameraProvider.getInstance(this)

    cameraProviderFuture.addListener(Runnable {
        // Used to bind the lifecycle of cameras to the lifecycle owner
        val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()

        // Preview
        val preview = Preview.Builder()
            .setTargetRotation(this.windowManager.defaultDisplay.rotation)
            .build()
            .also {
                it.setSurfaceProvider(viewFinder.createSurfaceProvider())
            }

        imageCapture = ImageCapture.Builder()
            .setTargetRotation(this.windowManager.defaultDisplay.rotation)
            .build()

        val imageAnalyzer = ImageAnalysis.Builder()
            .build()
            .also {
                it.setAnalyzer(cameraExecutor, LuminosityAnalyzer { luma ->
                    Log.d(TAG, "Average luminosity: $luma")
                })
            }

        // Select back camera as a default
        val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA

        try {
            // Unbind use cases before rebinding
            cameraProvider.unbindAll()

            // Bind use cases to camera
            cameraProvider.bindToLifecycle(
                this, cameraSelector, preview, imageCapture, imageAnalyzer)

        } catch(exc: Exception) {
            Log.e(TAG, "Use case binding failed", exc)
        }

    }, ContextCompat.getMainExecutor(this))
}

Here is the method to capture image:

private fun takePhoto() {
    val imageCapture = imageCapture ?: return

    // Create time-stamped output file to hold the image
    val photoFile = File(
        outputDirectory,
        SimpleDateFormat(FILENAME_FORMAT, Locale.US
        ).format(System.currentTimeMillis()) + ".jpg")

    // Create output options object which contains file + metadata
    val outputOptions = ImageCapture.OutputFileOptions.Builder(photoFile).build()

    // Set up image capture listener, which is triggered after photo has
    // been taken
    imageCapture.takePicture(
        outputOptions, ContextCompat.getMainExecutor(this), object : ImageCapture.OnImageSavedCallback {
            override fun onError(exc: ImageCaptureException) {
                Log.e(TAG, "Photo capture failed: ${exc.message}", exc)
            }

            override fun onImageSaved(output: ImageCapture.OutputFileResults) {
                val savedUri = Uri.fromFile(photoFile)
                val msg = "Photo capture succeeded: $savedUri"
                val bitmap = MediaStore.Images.Media.getBitmap(contentResolver, savedUri)
                ivCapturedImage.setImageBitmap(bitmap)
                setCaptureUI(false)
                Log.d(TAG, msg)
            }
        })
}

Do i need to rotate the image by myself after it is taken using EXIF or i can fix it while configuring camera?

Golconda

clarification question: Are you using a samsung device? Many of them have a known bug that logs the image at the wrong orientation when saved. – Bouffard

@JohnLord do you have any sources for this statement? Are there any known workarounds? – Kilbride

the only known workaround involves saving the image and then reading the exif data. It's a well-known issue and has various postings about it on stackoverflow, such as this one. #47261934 Our company has hundreds of samsung tablets and we had to include a similar fix to the link above, although ours was simpler because our tablets are locked to portrait. In the link above, they are comparing the read exif data to the current device orientation. – Bouffard

I thought with CameraX we won't have to work with exif data now? @JohnLord – Masturbation

I frankly don't know. A coworker did the conversion of our camera app to use camerax. I'm not even sure the problem was some kind of bug. Now you can pull properties from the camera sensor and this tells you that some of them are mounted 90 or 270 degrees so software that doesn't recognize this will most likely display the image wrong. – Bouffard

Answer 1

By default, ImageCapture set the orientation of the capture to the display rotation. If the image is saved to disk, the rotation will be in the EXIF.

Is your device in locked portrait mode? In that case, display rotation does not match the device's orientation, and you will need to set the target rotation yourself. Example.

// The value is whatever the display rotation should be, if the device orientation is not locked.
imageCapture.setTargetRotation(...) 

Or, you could simply use the LifecycleCameraController API. It handles the rotation for you and make all the use cases consistent in a WYSIWYG way.

Bart

A problem I have with LifecycleCameraController is that boilerplate code is missing; the one in the API description page is not enough to get started, and programming by trial and error is very time-consuming in Android apps... I'd really appreciate a codelab extension on this. I'll check the official cameraXsample to see if it helps. – Kilbride

I understand that the code lab is a little outdated. If you are interested, you can take a look at CameraX' test app on how the controller should be used: android.googlesource.com/platform/frameworks/support/+/refs/… – Pedanticism

Thanks for the link; but as far as I understand, CameraX does not like to meddle with image data so the only thing it does is set EXIF metadata. In the end I still used the ProcessCameraProvider class, set the target rotation just before calling takePicture(), then manually read the EXIF data from the saved JPEG, and rotated the picture accordingly to get a straight up image. I'll add this as a separate answer to the question, even though I'd like for CameraX to take care of rotating the image and not relying on EXIF metadata. – Kilbride

I tried to use LifecycleCameraController. And it didn't help. Photos are still rotated -90 degrees. And by the way, this class is very raw. There is no way to set the resolution for photo/analysis and other basic features are not available. – Autoeroticism

omg i love you for mentioning LifecycleCameraController. I deleted all my code and replaced it with the controller. – Cordilleras

Answer 2

I have used this class to rotate image

object CaptureImageHelper {

/**
 * This method is responsible for solving the rotation issue if exist. Also scale the images to
 * 1024x1024 resolution
 *
 * @param context       The current context
 * @param selectedImage The Image URI
 * @return Bitmap image results
 * @throws IOException
 */
@Throws(IOException::class)
fun handleSamplingAndRotationBitmap(context: Context, selectedImage: Uri?): Bitmap? {
    val MAX_HEIGHT = 1024
    val MAX_WIDTH = 1024

    // First decode with inJustDecodeBounds=true to check dimensions
    val options = BitmapFactory.Options()
    options.inJustDecodeBounds = true
    var imageStream: InputStream = context.getContentResolver().openInputStream(selectedImage!!)!!
    BitmapFactory.decodeStream(imageStream, null, options)
    imageStream.close()

    // Calculate inSampleSize
    options.inSampleSize = calculateInSampleSize(options, MAX_WIDTH, MAX_HEIGHT)

    // Decode bitmap with inSampleSize set
    options.inJustDecodeBounds = false
    imageStream = context.getContentResolver().openInputStream(selectedImage!!)!!
    var img = BitmapFactory.decodeStream(imageStream, null, options)
    img = rotateImageIfRequired(img!!, selectedImage)
    return img
}

/**
 * Calculate an inSampleSize for use in a [BitmapFactory.Options] object when decoding
 * bitmaps using the decode* methods from [BitmapFactory]. This implementation calculates
 * the closest inSampleSize that will result in the final decoded bitmap having a width and
 * height equal to or larger than the requested width and height. This implementation does not
 * ensure a power of 2 is returned for inSampleSize which can be faster when decoding but
 * results in a larger bitmap which isn't as useful for caching purposes.
 *
 * @param options   An options object with out* params already populated (run through a decode*
 * method with inJustDecodeBounds==true
 * @param reqWidth  The requested width of the resulting bitmap
 * @param reqHeight The requested height of the resulting bitmap
 * @return The value to be used for inSampleSize
 */
private fun calculateInSampleSize(
    options: BitmapFactory.Options,
    reqWidth: Int, reqHeight: Int
): Int {
    // Raw height and width of image
    val height = options.outHeight
    val width = options.outWidth
    var inSampleSize = 1
    if (height > reqHeight || width > reqWidth) {

        // Calculate ratios of height and width to requested height and width
        val heightRatio =
            Math.round(height.toFloat() / reqHeight.toFloat())
        val widthRatio =
            Math.round(width.toFloat() / reqWidth.toFloat())

        // Choose the smallest ratio as inSampleSize value, this will guarantee a final image
        // with both dimensions larger than or equal to the requested height and width.
        inSampleSize = if (heightRatio < widthRatio) heightRatio else widthRatio

        // This offers some additional logic in case the image has a strange
        // aspect ratio. For example, a panorama may have a much larger
        // width than height. In these cases the total pixels might still
        // end up being too large to fit comfortably in memory, so we should
        // be more aggressive with sample down the image (=larger inSampleSize).
        val totalPixels = width * height.toFloat()

        // Anything more than 2x the requested pixels we'll sample down further
        val totalReqPixelsCap = reqWidth * reqHeight * 2.toFloat()
        while (totalPixels / (inSampleSize * inSampleSize) > totalReqPixelsCap) {
            inSampleSize++
        }
    }
    return inSampleSize
}

/**
 * Rotate an image if required.
 *
 * @param img           The image bitmap
 * @param selectedImage Image URI
 * @return The resulted Bitmap after manipulation
 */
@Throws(IOException::class)
private fun rotateImageIfRequired(img: Bitmap, selectedImage: Uri): Bitmap? {
    val ei = ExifInterface(selectedImage.path)
    val orientation: Int =
        ei.getAttributeInt(ExifInterface.TAG_ORIENTATION, ExifInterface.ORIENTATION_NORMAL)
    return when (orientation) {
        ExifInterface.ORIENTATION_ROTATE_90 -> rotateImage(img, 90)
        ExifInterface.ORIENTATION_ROTATE_180 -> rotateImage(img, 180)
        ExifInterface.ORIENTATION_ROTATE_270 -> rotateImage(img, 270)
        else -> img
    }
}

private fun rotateImage(img: Bitmap, degree: Int): Bitmap? {
    val matrix = Matrix()
    matrix.postRotate(degree.toFloat())
    val rotatedImg =
        Bitmap.createBitmap(img, 0, 0, img.width, img.height, matrix, true)
    img.recycle()
    return rotatedImg
}

}

Golconda

Answer 3

I'm suffering from same situation. I solved this with hacky way.

My solution is:

    fun Bitmap.rotate(degrees: Float): Bitmap {
    val matrix = Matrix().apply { postRotate(degrees) }
    return Bitmap.createBitmap(this, 0, 0, width, height, matrix, true)
    }

Usage :

imageViewCapturedImage.setImageBitmap(bitmap?.rotate(90F))

Merc

Yes i am also rotating it myself. Please see my answer for rotation helper class – Golconda

Answer 4

The simplest solution that works for me.

Get the rotationDegrees from imageProxy and rotate your bitmap by that degree.

Matrix matrix = new Matrix();                   
matrix.postRotate((float)imageProxy.getImageInfo().getRotationDegrees());
Bitmap bitmap2 =  Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
binding.imgPreview.setImageBitmap(bitmap2);

Sawicki

Answer 5

This simple code worked for me:

Java Version:

Context context = ... //The current Context
Camera camera = cameraProvider.bindToLifecycle(...); //The one you get after initializing the camera
ImageProxy image = ... //The one that takePicture or Analyze give you
int currentLensOrientation = ... //CameraSelector.LENS_FACING_BACK or CameraSelector.LENS_FACING_FRONT

int rotationDirection = currentLensOrientation == CameraSelector.LENS_FACING_BACK ? 1 : -1;
int constantRotation = image.getImageInfo().getRotationDegrees() - camera.getCameraInfo().getSensorRotationDegrees();
int rotationDegrees = camera.getCameraInfo().getSensorRotationDegrees() - context.getDisplay().getRotation() * 90 + constantRotation * rotationDirection;

Kotlin Version:

val context: Context = ... //The current Context
val camera: Camera? = cameraProvider.bindToLifecycle(...) //The one you get after initializing the camera
val image: ImageProxy = ... //The one that takePicture or Analyze give you
val currentLensOrientation: Int = ... //CameraSelector.LENS_FACING_BACK or CameraSelector.LENS_FACING_FRONT

val rotationDirection = if (currentLensOrientation == CameraSelector.LENS_FACING_BACK) 1 else -1
val constantRotation = image.imageInfo.rotationDegrees - camera!!.cameraInfo.sensorRotationDegrees
val rotationDegrees = camera!!.cameraInfo.sensorRotationDegrees - context.display!!.rotation * 90 + constantRotation * rotationDirection

Then I used the rotationDegrees to rotate the ImageProxy that CameraX passes you in the takePicture and analyze's callbacks.

Here you can find the full Java code if you need it: https://github.com/CristianDavideConte/SistemiDigitali/blob/7b40e50d8b2fbdf4e4a61edba7443da92b96c58d/app/src/main/java/com/example/sistemidigitali/views/CameraProviderView.java#L207

Hill

Answer 6

I've been having the same problem; as far as I can understand, from replies on tickets such as this or this, the team behind CameraX does not like to meddle with raw image data as returned from the hardware, and would very much like to just limit themselves to setting EXIF metadata.

So I just worked around this, and starting from a code quite similar to yours (well, heavily inspired from the one in the codelab), I added this:

Display d = getDisplay();
if (d != null) {
    iCapture.setTargetRotation(d.getRotation());
}

just before calling iCapture.takePicture() (iCapture being my ImageCapture use-case instance). This ensures that the target rotation in the EXIF file metadata will be consistent with actual display rotation when the photo is taken.

Then, after data is received (in my case, on the onImageSaved() handler), I check the EXIF metadata for rotation and, in that case, manually rotate the image for the required degrees and save a different file to ensure no EXIF tags are kept with incoherent values.

try {
    ExifInterface ei = new ExifInterface(tempFile.getAbsolutePath());
    if (ei.getRotationDegrees() != 0) {
        actualPicture = ImageUtil.rotateDegrees(tempFile, ei.getRotationDegrees());
    }
} catch (IOException exc) {
    Log.e(TAG, "Tried to fix image rotation but could not continue: " + exc,getMessage());
}

where ImageUtil is a custom class of image tools, and rotateDegrees does just that, with a custom matrix initialized like this:

//inside rotateDegrees(), degrees is the parameter to the function
Matrix m = new Matrix();
m.postRotate(degrees);

and creating a new bitmap starting from the one imported from the original file:

Bitmap b = Bitmap.createBitmap(sourceFile, 0, 0, sourceFile.getWidth(), sourceFile.getHeight(), m, true);
b.compress(Bitmap.CompressFormat.JPEG, 85, /* a suitably-created output stream */);

Still, I'd like for CameraX to handle image rotation directly, without relying on metadata (that, by their own admission, very few libraries and tools go read and actually handle).

Kilbride

Only working with >= Build.VERSION_CODES.R – Apnea