I'm looking for help with an issue I'm facing using Grafika's CameraCaptureActivity code. I want to build an app that can record camera and display a preview, so this sample and code looked like exactly what I wanted and so far it's been great, this issue appart.
The issue I have is that when the camera preview size doesn't match the exact size of the GLSurfaceView I use to display the preview it shows a stretched preview.
Here's how it looks:
As you can see it's stretched horizontally since the note sheet is a perfect square.
For this precise exemple on a Samsung Galaxy S4, the camera preview size is 1280x720 (extracted from available preview sizes using Camera2 API), and the GLSurfaceView is 1920x1080 to cover the entire screen.
When does it appear?
The issue appears when the SurfaceView is upscaled compared to the Camera output, even if I always make sure the ratio remains correct (1920x1080 = 1,5 * 1280x720).
What code are you using?
I use Grafika's code, including:
Some log?
Here's what SurfaceFlinger dump looks like:
Hardware Composer state (version 01030000):
mDebugForceFakeVSync=0
Display[0] configurations (* current):
* 0: 1080x1920, xdpi=442.450989, ydpi=439.351013, secure=1 refresh=16666667
numHwLayers=3, flags=00000000
type | handle | hint | flag | tr | blnd | format | source crop(l,t,r,b) | frame | dirtyRect | name
------------+----------+----------+----------+----+-------+----------+-----------------------------------+---------------------------+-------------------
HWC | b3b12ba0 | 0002 | 0000 | 00 | 0100 | RGB_888 | 0.0, 0.0, 1080.0, 1920.0 | 0, 0, 1080, 1920 | [ 0, 0, 1080, 1920] | SurfaceView
HWC | b3d12740 | 0002 | 0000 | 00 | 0105 | RGBA_8888 | 0.0, 0.0, 1065.0, 1905.0 | 0, 0, 1065, 1905 | [ 0, 0, 1065, 1905] | com.mybundle.myapp/com.mybundle.myapp.activity.MyActivity
FB TARGET | b6a55c40 | 0000 | 0000 | 00 | 0105 | RGBA_8888 | 0.0, 0.0, 1080.0, 1920.0 | 0, 0, 1080, 1920 | [ 0, 0, 0, 0] | HWC_FRAMEBUFFER_TARGET
Layer 0xb4094000 (SurfaceView)
Region transparentRegion (this=0xb4094160, count=1)
[ 0, 0, 0, 0]
Region visibleRegion (this=0xb4094008, count=1)
[ 0, 0, 1920, 1080]
layerStack= 0, z= 21015, pos=(0,0), size=(1920,1080), crop=( 0, 0,1920,1080), isOpaque=1, invalidate=0, alpha=0xff, flags=0x00000002, tr=[1.00, 0.00][0.00, 1.00]
client=0xb66190c0
format= 4, activeBuffer=[1080x1920:1536, 3], queued-frames=0, mRefreshPending=0
mTexName=116 mCurrentTexture=0
mCurrentCrop=[0,0,0,0] mCurrentTransform=0x7
mAbandoned=0
-BufferQueue mMaxAcquiredBufferCount=1, mDequeueBufferCannotBlock=0, default-size=[1920x1080], default-format=4, transform-hint=04, FIFO(0)={}
>[00:0xb1c18500] state=ACQUIRED, 0xb3b12ba0 [1080x1920:1536, 3]
Camera preview size code?
StreamConfigurationMap map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
// Retrieve display size
WindowManager wm = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE);
Display display = wm.getDefaultDisplay();
Point displaySize = new Point();
display.getSize(displaySize);
// Choose the sizes for camera preview
mPreviewSize = chooseOptimalSize(map.getOutputSizes(SurfaceTexture.class),
displaySize.x, displaySize.y, captureWidth, captureHeight);
private static Size chooseOptimalSize(Size[] choices, int screenWidth, int screenHeight, int captureWidth, int captureHeight)
{
// Collect the supported resolutions that are at least as big as the screen && wanted capture size
List<Size> bigEnough = new ArrayList<>();
// Collect the supported resolutions that are at least as big as the wanted capture size but < to screensize
List<Size> smallerButEnough = new ArrayList<>();
for (Size option : choices) {
if (option.getHeight() == option.getWidth() * captureHeight / captureWidth) {
if( option.getWidth() >= screenWidth && option.getHeight() >= screenHeight )
{
bigEnough.add(option);
}
else
{
smallerButEnough.add(option);
}
}
}
// Pick the smallest of those, assuming we found any
if (bigEnough.size() > 0)
{
return Collections.min(bigEnough, new CompareSizesByArea());
}
// Pick the biggest of those, assuming we found any
else if( smallerButEnough.size() > 0 )
{
return Collections.max(smallerButEnough, new CompareSizesByArea());
}
else
{
Log.e(TAG, "Couldn't find any suitable preview size");
return choices[0];
}
}
What have you tried?
I first thought it was a Camera2 issue, so I tried to use only the old Camera API, but result is the same.
Then I thought it was a ratio issue, but as you can see the ratio here is ok horizontally and vertically.
My bet is that there's an OpenGL transformation to apply when the Surface is resized but I'm a pure noob in OpenGL so I haven't been able to find anything that works. I try added those lines but it doesn't help:
@Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
Log.d(TAG, "onSurfaceChanged " + width + "x" + height);
gl.glViewport(0, 0, width, height);
gl.glClear(GLES10.GL_COLOR_BUFFER_BIT | GLES10.GL_DEPTH_BUFFER_BIT);
}
Another example, right from Grafika:
Here's an other example of the issue, right from the Grafika app.
Phone: Galaxy S3 Neo (GT-I9301L) - Android 4.4.2
The phone has a 1280x720 screen, camera opened with the 640x480 resolution, displayed in a 573x430. Here's how it looks:
SurfaceFlinger logs:
Hardware Composer state (version 1030000):
mDebugForceFakeVSync=0
Display[0] : 720x1280, xdpi=304.799011, ydpi=306.716003, refresh=16666667
numHwLayers=4, flags=00000000
type | handle | hints | flags | tr | blend | format | source crop | frame name
------------+----------+----------+----------+----+-------+----------+---------------------------------+--------------------------------
HWC | b84a3d18 | 00000002 | 00000000 | 00 | 00100 | 00000002 | [ 0.0, 0.0, 430.0, 573.0] | [ 32, 353, 462, 926] SurfaceView
HWC | b841b080 | 00000002 | 00000000 | 00 | 00105 | 00000001 | [ 0.0, 0.0, 670.0, 1280.0] | [ 0, 0, 670, 1280] com.android.grafika/com.android.grafika.CameraCaptureActivity
HWC | b8423250 | 00000002 | 00000000 | 00 | 00105 | 00000001 | [ 0.0, 0.0, 50.0, 1280.0] | [ 670, 0, 720, 1280] StatusBar
FB TARGET | b8412d50 | 00000000 | 00000000 | 00 | 00105 | 00000001 | [ 0.0, 0.0, 720.0, 1280.0] | [ 0, 0, 720, 1280] HWC_FRAMEBUFFER_TARGET
+ Layer 0xb8416080 (SurfaceView) id=112
Region transparentRegion (this=0xb84169d0, count=1)
[ 0, 0, 0, 0]
Region visibleRegion (this=0xb8416088, count=1)
[353, 258, 926, 688]
layerStack= 0, z= 21015, pos=(353,258), size=( 573, 430), crop=( 0, 0, 573, 430), isOpaque=1, invalidate=0, alpha=0xff, flags=0x00000000, tr=[1.00, 0.00][0.00, 1.00]
client=0xb841d7c8
format= 4, activeBuffer=[ 430x 573: 448, 2], queued-frames=0, mRefreshPending=0
mTexName=95 mCurrentTexture=1
mCurrentCrop=[0,0,0,0] mCurrentTransform=0x7
mAbandoned=0
-BufferQueue mMaxAcquiredBufferCount=1, mDequeueBufferCannotBlock=0, default-size=[573x430], default-format=4, transform-hint=04, FIFO(0)={}
[00:0xb84439a8] state=FREE , 0xb849a528 [ 430x 573: 448, 2]
>[01:0xb8421128] state=ACQUIRED, 0xb84a3d18 [ 430x 573: 448, 2]
[02:0xb84238e8] state=FREE , 0xb84432f8 [ 430x 573: 448, 2]
Grafika's app log:
04-18 18:06:49.954 14383-14383/com.android.grafika D/Grafika: onCreate complete: com.android.grafika.CameraCaptureActivity@42adc288
04-18 18:06:49.954 14383-14383/com.android.grafika D/Grafika: onResume -- acquiring camera
04-18 18:06:50.064 14383-14383/com.android.grafika D/Grafika: Camera preferred preview size for video is 1920x1080
04-18 18:06:50.064 14383-14383/com.android.grafika D/Grafika-AFL: Setting aspect ratio to 1.3333333333333333 (was -1.0)
04-18 18:06:50.064 14383-14383/com.android.grafika D/Grafika: onResume complete: com.android.grafika.CameraCaptureActivity@42adc288
04-18 18:06:50.064 14383-32312/com.android.grafika D/Grafika: setCameraPreviewSize
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 526]
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: new size=701x526 + padding 0x0
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 430]
04-18 18:06:50.094 14383-14383/com.android.grafika D/Grafika-AFL: new size=573x430 + padding 0x0
04-18 18:06:50.124 14383-32312/com.android.grafika D/Grafika: onSurfaceCreated
04-18 18:06:50.134 14383-32312/com.android.grafika D/Grafika: Created program 3 (TEXTURE_EXT)
04-18 18:06:50.144 14383-32312/com.android.grafika D/Grafika: onSurfaceChanged 573x430
04-18 18:06:50.144 14383-32312/com.android.grafika D/Grafika: Updating filter to 0
04-18 18:06:50.154 14383-14383/com.android.grafika D/Grafika: onItemSelected: 0
04-18 18:06:50.154 14383-14383/com.android.grafika D/Grafika: CameraHandler [Handler (com.android.grafika.CameraCaptureActivity$CameraHandler) {42aeec70}]: what=0
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 526]
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: new size=701x526 + padding 0x0
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: onMeasure target=1.3333333333333333 width=[MeasureSpec: EXACTLY 1216] height=[MeasureSpec: EXACTLY 430]
04-18 18:06:50.514 14383-14383/com.android.grafika D/Grafika-AFL: new size=573x430 + padding 0x0
04-18 18:06:50.544 14383-14383/com.android.grafika D/AbsListView: onVisibilityChanged() is called, visibility : 4
04-18 18:06:50.544 14383-14383/com.android.grafika D/AbsListView: unregisterIRListener() is called
So...
If one of you guys have an idea of how to make it work, I would love to hear from him.
Thanks!
adb shell dumpsys SurfaceFlingerto dump the HardwareComposer layers; this will show the transformation that's being applied to the SurfaceView Surface. source.android.com/devices/graphics/… has an example (using a video player from Grafika) of the section you should look at. GLES transforms shouldn't be relevant since you're just drawing the entire frame with a viewport-sized rect. Do you see any distortion in Grafika itself? – Chingadb shell dumpsys SurfaceFlingeron my answer: #36617782. Do you see anything weird? – Reconstructionistcamera2code for preview size. Thanks for your help – ReconstructionistmPreviewSize.getWidth()/mPreviewSize.getHeight()? – Ching1280, height:720. – ReconstructionistopenCamera(640, 480);, all the rest is unchanged. Does this other example tells us something? @EddyTalvala I'm not getting anything on the Galaxy S3 running 4.4.1:Client[0] (0xb71a8178) PID: 14383is the only non-static log. – ReconstructionistonSurfaceChanged()... does that match expectations? When you try to run Grafika at 640x480, do you see an "Unable to set preview size" complaint in the log? (I'm wondering if your camera doesn't support that resolution, and Grafika is stupidly not passing the error out ofchoosePreviewSize().) – ChingopenCamera(640, 480)). I see the samenew size=573x430log. On my Moto X running 4.4.4, squares look square. I grabbed a screen shot for comparison (imgur.com/1oJNXYH). Our screens don't line up exactly, but that might be because my device has a visible nav bar on the right. Measuring your image with GIMP shows a 16:9 screen shot with a 4:3 camera area, so the View layout is fine. We seem to be doing the same thing, but for some reason it looks wrong on your device and fine on mine. – ChingopenCamera()tries for CAMERA_FACING_BACK instead? (Wondering if the device has two cameras, and the data for one is wrong, but nobody notices because pictures are usually taken with the back-facing camera.) – ChingcaptureWidth&captureHeight? 2) Are you doingsetPreviewSize()andsetPictureSizeto the samemPreviewSize? 3) Do you know if the problem devices might not be 16x9? – Exceeding