android textureview full screen preview with correct aspect ratio

Question

I have been working with the camera2 api demo from google and unfortunately the sample application is built to display the textureview preview at approximately 70% of the screen height, after looking around I was able to determine that this was being caused by the AutoFitTextureView overriding the onMeasure() method as shown below:

@Override
protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
    super.onMeasure(widthMeasureSpec, heightMeasureSpec);
    int width = MeasureSpec.getSize(widthMeasureSpec);
    int height = MeasureSpec.getSize(heightMeasureSpec);
    if (0 == mRatioWidth || 0 == mRatioHeight) {
        setMeasuredDimension(width, height);
    } else {
        if (width < height * mRatioWidth / mRatioHeight) {
            setMeasuredDimension(width, width * mRatioHeight / mRatioWidth);
        } else {
            setMeasuredDimension(height * mRatioWidth / mRatioHeight, height);
        }
    }
}

I attempted to correct this by setting the correct heights and widths in setMeasuredDimension(width, height);, this fixed the height issue and gave me a full screen preview from the textureview, however the aspect ratio is completely broken and warped on every device, what is the standard way of fixing this? I see many apps on the play store have found a way to solve this problem but havent been able to track down a fix, any help will go a long way, thanks.

Ultrared

Hey you found any solution? facing the same problem.. please help. – Bainite

Answer updated below – Ultrared

Answer 1

I was able to fix the issue by switching setMeasuredDimension();

    int width = MeasureSpec.getSize(widthMeasureSpec);
    int height = MeasureSpec.getSize(heightMeasureSpec);
    if (0 == mRatioWidth || 0 == mRatioHeight) {
        setMeasuredDimension(width, height);
    } else {
        if (width < height * mRatioWidth / mRatioHeight) {
            setMeasuredDimension(height * mRatioWidth / mRatioHeight, height);
        } else {
            setMeasuredDimension(width, width * mRatioHeight / mRatioWidth);
        }
    }

Ultrared

Hello Edmund, What do you mean by "switching setMeasuredDimension()"? I have the same issue with full screen with camera2. Could you provide the code you use in the onMeasure method to get the full screen working without any distortion please? – Kc

Sorry for the late response, I have updated my answer – Ultrared

Thanks for the updated answer but this solution stretches the preview. – Bainite

Thanks for the solution. You should mark it as answer. – Sibilla

thanks this works... should be marked as correct answer – Slusher

The switching does seem to work but a little bit of explanation on why will be really helpful. Also, this knocks the FrameLayout view out of the screen in the Camera2Raw application code. – Salgado

Don't forget to set the view height and width to wrap_content in xml – Bedplate

@Bedplate this thing work for me i was frustrated because i used constraint Layout and set constraint height width so it will stretched my preview. mark-able answer... +1 – Archibald

Answer 2

Below is the code that we used to measure a preview which supports 4:3, 16:9 and 1:1 preview sizes. The height is scaled because the app is block in portrait, it does not rotate to landscape.

protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
    super.onMeasure(widthMeasureSpec, heightMeasureSpec);
    int width = MeasureSpec.getSize(widthMeasureSpec);
    int height = MeasureSpec.getSize(heightMeasureSpec);

    Log.d(TAG, "[onMeasure] Before transforming: " + width + "x" + height);

    int rotation = ((Activity) getContext()).getWindowManager().getDefaultDisplay().getRotation();
    boolean isInHorizontal = Surface.ROTATION_90 == rotation || Surface.ROTATION_270 == rotation;

    int newWidth;
    int newHeight;

    Log.d(TAG, "[onMeasure] Get measured dimensions: " + getMeasuredWidth() + "x" + getMeasuredHeight());

    if (isInHorizontal) {
        newHeight = getMeasuredHeight();
        if (mAspectRatioOneOne) newWidth = getMeasuredHeight();
        else newWidth = (int) (newHeight * mAspectRatio);
    } else {
        newWidth = getMeasuredWidth();
        if (mAspectRatioOneOne) newHeight = getMeasuredWidth();
        else newHeight = (int) (newWidth * mAspectRatio);
    }

    setMeasuredDimension(newWidth, newHeight);
    Log.d(TAG, "[onMeasure] After transforming: " + getMeasuredWidth() + "x" + getMeasuredHeight());

}

Arlinearlington

Answer 3

I had similar problem using Camera2 implementation. I used AutoFitTextureView implementation of TextureView as preview from camera, and want to adjust to correct ratio. My problem income from different implementations of methods and propagate to onSurfaceTextureSizeChanged. [1]com.google.android.cameraview.CameraView#onMeasure(..) -> [2]com.google.android.cameraview.AutoFitTextureView#onMeasure(..) -> [3]onSurfaceTextureSizeChanged(..)

Problem where in different if [1] than in [2]. I replaced in [1]:

if (height < width * ratio.getY() / ratio.getX()) {

to

if (!(height < width * ratio.getY() / ratio.getX())) {

because [2] had if based on width not height

 if (width < height * mRatioWidth / mRatioHeight) {

Verify if methods onMeasure(..) are implemented correctly.

Or you could do like Edmund Rojas - above

Reade

Answer 4

If you want full-screen, undistorted preview, then you have to select a preview resolution for the camera that matches the aspect ratio of your screen.

However, if your screen size isn't a standard size (16:9 or 4:3, basically), especially once you subtract off the soft buttons and the notification bar (assuming you're not completely full screen), then the only option for having the preview fill the screen is to cut some of it off.

You should be able to modify the TextureView's transform matrix to remove the distortion, by looking at the view's width and height and the selected preview width and height, but this will necessarily cut some of it off.

Homozygote

Hey, how would you go and "zoom in", to keep the aspect ratio but fill the whole screen with the camera preview? You kinda describe it in your last paragraph, but I don't understand how to modify the transform matrix correctly. – Chenee

The transform matrix is a standard graphics 4x4 matrix in homogeneous coordinates. To fill the screen, you just need to uniformly scale up the image until the black bars are gone (note that this will put some image data off-screen). This is from memory, but the scale matrix is just [s 0 0 0; 0 s 0 0; 0 0 0 0; 0 0 0 1] where s is the scale-up factor (so s=2, for double size). Multiply that matrix with the transform matrix you get from getTransform (matrix multiply, not elementwise), and set it as the transform with setTransform. – Homozygote

Alright, I understood the matrix.setScale(x, y) method. I will try out this "multipy that matrix with the transform matrix..."-part tomorrow and see if i get it to work. Im not 100% sure what you mean by that. I might hit u up again, thanks alot so far! – Chenee

How would I calculate the X and Y scale of the transform matrix, so it fits for either screen orientation (landscape/portrait)? I have the screen width + height and the preview width + height. I would somehow have to get values for X and Y (depending on Orientation) where one of them is "1" and the other is slightly bigger, around 1,2 most of the times. I'm setting the transform when surface changes callback is fired btw. – Chenee

Mhhh wait, this "either one value is 1" might be wrong – Chenee

Thank you sir, I have accomplished it, I even had an open question on SO for this, found here: #49172085 – Chenee

Answer 5

AutoFitTextureView in layout should use wrap_content

    <AutoFitTextureView
        android:id="@+id/texture"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"/>

/**
 * A [TextureView] that can be adjusted to a specified aspect ratio.
 */
class AutoFitTextureView : TextureView {

    constructor(context: Context) : this(context, null)
    constructor(context: Context, attrs: AttributeSet?) : this(context, attrs, 0)
    constructor(context: Context, attrs: AttributeSet? = null, defStyle: Int = 0) : super(context, attrs, defStyle)

    private var ratioWidth = 0
    private var ratioHeight = 0

    /**
     * Sets the aspect ratio for this view. The size of the view will be measured based on the ratio
     * calculated from the parameters. Note that the actual sizes of parameters don't matter, that
     * is, calling setAspectRatio(2, 3) and setAspectRatio(4, 6) make the same result.
     *
     * @param width  Relative horizontal size
     * @param height Relative vertical size
     */
    fun setAspectRatio(width: Int, height: Int) {
        require(!(width < 0 || height < 0)) { "Size cannot be negative." }
        ratioWidth = width
        ratioHeight = height
        requestLayout()
    }

    override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
        super.onMeasure(widthMeasureSpec, heightMeasureSpec)
        val width = MeasureSpec.getSize(widthMeasureSpec)
        val height = MeasureSpec.getSize(heightMeasureSpec)
        if (ratioWidth == 0 || ratioHeight == 0) {
            setMeasuredDimension(width, height)
        } else {
            if (width < height * ratioWidth / ratioHeight) {
                setMeasuredDimension(width, width * ratioHeight / ratioWidth)
            } else {
                setMeasuredDimension(height * ratioWidth / ratioHeight, height)
            }
        }
    }

}

Vowell