In stereo calibration, how the extrinsic matrix changes if i change the resolution of stereo camera

Question

My stereo camera has different resolutions 1280x480, 640x240 and 320x120. (The camera stream a synchronized pair of images 640X480 pasted horizontally that's why is 1280x480).

I have used the algorithm of Opencv3 Stereo Calibration in the following link to calibrate the stereo camera with the resolution of 1280*480.

    stereoRectify( M1, D1, M2, D2, img_size, R, T, R1, R2, P1, P2, Q, CALIB_ZERO_DISPARITY, -1, img_size, &roi1, &roi2 );
    Mat map11, map12, map21, map22;
    initUndistortRectifyMap(M1, D1, R1, P1, img_size, CV_16SC2, map11, map12);
    initUndistortRectifyMap(M2, D2, R2, P2, img_size, CV_16SC2, map21, map22);

    Mat img1r, img2r;
    remap(img1, img1r, map11, map12, INTER_LINEAR);
    remap(img2, img2r, map21, map22, INTER_LINEAR);

The stereoRectify computes the rotation matrix R, translation matrix T between left and right camera. And also, it calculates two matrices of rotation R1, R2 and two matrices of projection P1 P2. I used the output of stereoRectify as the input of initUndistortRectifyMap and then remap to apply the projection.

Here is a stackoverflow answer to explain how to do it.

Now I have got the two matrices of rectification map of left map11, map12 and right camera map21, map22.

But now I want to use these camera matrices M1 and M2, distortion matrices D1 and D2, and extrinsic matrices R, T, R1, R2, P1 and P2 to rectify the camera images in a lower resolution (320x120) each. PS I haven't calibrated the camera with the resolution of 320*120 directly because the image is too small and the algorithm of Opencv can't find the chessboard corners to perform the calibration.

I know that "The distortion coefficients do not depend on the scene viewed. Thus, they also belong to the intrinsic camera parameters. And they remain the same regardless of the captured image resolution. If, for example, a camera has been calibrated on images of 320 x 240 resolution, absolutely the same distortion coefficients can be used for 640 x 480 images from the same camera while f_x, f_y, c_x, and c_y need to be scaled appropriately." according the documentation of opencv. ( I tested and it is working)

I want to know: how should I modify the matrices of R, T, R1, R2, P1, P2 to do the remap in a lower resolution from 1280x480 to 320x120?.

Mccarty

Answer 1

Actually I change the matrices of P1 and P2 which are the combination of camera intrinsic matrix and camera translation matrix.

I just divide them by 4 for using 320x120. The general formula is:

fx' = (dimx' / dimx) * fx

fy' = (dimy' / dimy) * fy

fx' is the value for your new resolution, fx is the value you already have for your original resolution, dimx' is the new resolution along the x axis, dimx is the original one. The same applies to fy.

cx and cy are calculated analogically because all these values are expressed in pixel coordinates.

Mccarty

Answer 2

After much playing around not only P1 and P2 should be scaled, but also M1 and M2 (in the OP's code).

Remember, that scaling is only two first diagonal entries of left scaling matrix, and preserve the third one to be 1.

Gayden