Camera2 replacing one logical stream with two physical streams in Android API 29

Question

When Android 9(API 28) was released, I was very happy to discover that the physical cameras of the phones with multiple cameras would be exposed, I had been very frustrated to not be able to access them. Today I stumbled upon the android Q documentation which says:

Starting from API level 29, some or all physical cameras may not be independently exposed to the application, in which case the physical camera IDs will not be available in CameraManager.getCameraIdList(). But the application can still query the physical cameras' characteristics by calling CameraManager.getCameraCharacteristics(String).

This statement confuses me, does it mean that Android is backtracking? what's the purpose of the change?

I'm interested in managing cameras at a very fine level, will this change prevent me from doing what I'm able to do today with Android 9?

It says that the IDs may not be listed any more but that the characteristics can still be queried, how are we supposed to get the characteristics of cameras without having their IDs? I guess we are supposed to get the physical cameras' IDs via getPhysicalCameraIds() on logical cameras, but does it means that if we wanted to use a "hidden" camera, we'd have to test a bunch of random ID strings?

And will we still be able to create capture sessions with SessionConfiguration on physical cameras that are not exposed?

Can someone shed some light on this?

Willamina

did you find a solution to use a specific camera from multicamera? e.g. the widest one? I guess it has something to do with camChars.availablePhysicalCameraRequestKeys and setPhysicalCameraKey – Fenestra

Answer 1

This is not backtracking. Actually, this generalization is an important step to provide you with better control of complex camera setups.

Before API 29, if the manufacturer could not provide separate cameraDevice for each of the lenses that constitute a composite camera, they had no way to expose the lens parameters to the developer. Their only choice was to expose the composite camera as whole.

The question is not

And will we still be able to create capture sessions with SessionConfiguration on physical cameras that are not exposed?

but rather,

And now we can read the characteristics of physical cameras that could not be exposed before‼

With the new change, the manufacturer can give you all available info on "Back camera1", "Back camera2", etc. even if there is no way to start separate capture sessions on them.

getPhysicalCameraIds() reference explains:

Prior to API level 29, all returned IDs are guaranteed to be returned by CameraManager.getCameraIdList(), and can be opened directly by CameraManager.openCamera(). Starting from API level 29, for each of the returned ID, if it's also returned by CameraManager.getCameraIdList(), it can be used as a standalone camera by CameraManager.openCamera(). Otherwise, the camera ID can only be used as part of the current logical camera.

This means that you can use any id returned by getPhysicalCameraIds() in CameraManager.getCameraCharacteristics(id):

From API level 29, this function can also be used to query the capabilities of physical cameras that can only be used as part of logical multi-camera. These cameras cannot not be opened directly via openCamera(String, CameraDevice.StateCallback, Handler).

Ironic

Oh I see, my error was to think that in Android 9 all the physical cameras were exposed. I was afraid it would be a way to let the brands block the access to the devices from third party developers. I had missed the part in getCameraIdList that says: "This list doesn't contain physical cameras that can only used as part of a logical multi-camera device." (missing "be" in that sentence) When everything is pieced together it makes sense. – Willamina

The double negative typo in CameraManager.getCameraCharacteristics(id) (...These cameras cannot not be opened directly via openCamera(String, CameraDevice.StateCallback, Handler)) adds to the confusion. – Willamina

you can open a bug against the CameraManager documentation ;) Also, in CameraCharacteristics they use this strange # notation which is result of lasy copy/paste – Ironic

so how to configure camera2 api to use wide camera? I see there is setPhysicalCameraKey but camChars.availablePhysicalCameraRequestKeys returns null so I cannot use setPhysicalCameraKey, only camChars.physicalCameraIds returns list with 3 ids – Fenestra

so does it mean that Pixel 5 allow to use only two cameras for other apps, though it has 3 physicals cameras for back camera? so all those physical cameras are only available for stock camera app? – Pinsk

@Pinsk does camChars.physicalCameraIds for Pixel 5 return only 2? – Ironic

@AlexCohn Pixel 5 returns two logical camera ids (back and front) cameraManager.cameraIdList, for first logical back camera it includes two physical camera ids cameraBackCharacteristics.physicalCameraIds, for front logical camera it retunes empty list, check my screenshot https://mcmap.net/q/1735561/-camera2-replacing-one-logical-stream-with-two-physical-streams-in-android-api-29 – Pinsk

@AlexCohn I updated my answer how to open physical camera after opening logical camera – Pinsk

Answer 2

I tried to open camera using physical ids of first logical back camera of Pixel 5 (Android 11) but it failed with error: unknown camera id....

So Pixel 5 returns only 2 logical camera ids and only those 2 cameras can be opened using cameraManager.openCamera(...) method...

Samsung S10 (also Android 11) returns 4 logical camera ids and doesn't have any physical camera ids for all those logical cameras, all 4 cameras can be used without any problem.

Pixel 5 camera info:

Samsung S10 camera info:

Though for Pixel 5 I didn't try setPhysicalCameraId(cameraId) https://developer.android.com/reference/android/hardware/camera2/params/OutputConfiguration.html#setPhysicalCameraId(java.lang.String)

I think this method can be used to switch between different physical camera (for example we can open wide lens camera)

Update

Yes we can use setPhysicalCameraId method with Pixel 4, 5 (and I guess there more devices and manufactures) to set specific physical camera

So logic is the next:

1. you open camera using logical camera id using camera manager

2. then if that logical camera supports multi camera feature then it should have non empty physical camera id list

3. you can choose any id from physical camera id list and set it to OutputConfiguration:

val outputs = surfaces.map {
    OutputConfiguration(it).apply {
        setPhysicalCameraId(cameraIdx)
    }
}
cameraDevice.createCaptureSessionByOutputConfigurations(
    outputs,
    stateCallback,
    backgroundCameraHandler
)

UPDATE! WARNING! but it previews the same view, so it seems this method doesn't work (though there is no exception), nothing is changed, whether you set physical camera or not

Seems it's broken starting from Android 10 - Android 10 (api 29) camera2 api regression with wide-angle camera

Update

I also tried to create capture session using SessionConfiguration object instead of deprecated method createCaptureSessionByOutputConfigurations

val config = SessionConfiguration(
    SessionConfiguration.SESSION_REGULAR,
    outputs,
    backgroundCameraExecutor!!,
    stateCallback
)
cameraDevice.createCaptureSession(config)

But still nothing is changed

Pinsk

Thanks for these updates, they are most appreciated even though they don't fit the scope of SO answer. – Ironic

Samsung Note 20 ultra and Samsung z fold 2 also gives 4 cameraid list 0 for rear normal(wide),1 for front, 2 for ultra wide camera and 3 for front camera again. Both devices behave same and for cameraid list. So, i want to ask if all samsung devices which has ultra wide camera behaves same(open ultra wide camera for cameraid-2). – Microscopium

On Pixel 5, you can enable the "ultrawide" camera by setting the CONTROL_ZOOM_RATIO to the lowest supported value (0.615), see my answer for more details. – Rubious

Answer 3

On Pixel 5, you can enable the "ultrawide" camera by setting the CONTROL_ZOOM_RATIO to the lowest supported value (0.615), see here https://tech.gc.com/every-camera-every-angle-on-android/ .

Changing the physical camera ID, which is also possible through different ill-documented methods (e.g., ACaptureSessionPhysicalOutput_create in the C API), is not sufficient (nor necessary) to get a wide image, since the with default zoom (1.0), the image from the wide cam will be digitally zoomed.

Rubious