Field of view of a GoPro camera

Question

I have calibrated my GoPro Hero 4 Black using Camera calibration toolbox for Matlab and calculated its fields of view and focal length using OpenCV's calibrationMatrixValues(). These, however, differ from GoPro's specifications. Istead of 118.2/69.5 FOVs I get 95.4/63.4 and focal length 2.8mm instead of 17.2mm. Obviously something is wrong.

I suppose the calibration itself is correct since image undistortion seems to be working well.

Can anyone please give me a hint where I made a mistake? I am posting my code below.

Thanks.

Code

cameraMatrix = new Mat(3, 3, 6);
for (int i = 0; i < cameraMatrix.height(); i ++)
    for (int j = 0; j < cameraMatrix.width(); j ++) {
    cameraMatrix.put(i, j, 0);
}
cameraMatrix.put(0, 0, 582.18394);
cameraMatrix.put(0, 2, 663.50655);
cameraMatrix.put(1, 1, 582.52915);
cameraMatrix.put(1, 2, 378.74541);
cameraMatrix.put(2, 2, 1.);

org.opencv.core.Size size = new org.opencv.core.Size(1280, 720);
//output parameters
double [] fovx = new double[1];
double [] fovy = new double[1];
double [] focLen = new double[1];
double [] aspectRatio = new double[1];
Point ppov = new Point(0, 0);
org.opencv.calib3d.Calib3d.calibrationMatrixValues(cameraMatrix, size,
        6.17, 4.55, fovx, fovy, focLen, ppov, aspectRatio);


System.out.println("FoVx: " + fovx[0]);
System.out.println("FoVy: " + fovy[0]);
System.out.println("Focal length: " + focLen[0]);
System.out.println("Principal point of view; x: " + ppov.x + ", y: " + ppov.y);
System.out.println("Aspect ratio: " + aspectRatio[0]);

Results

FoVx: 95.41677635378488
FoVy: 63.43170132212425
Focal length: 2.8063085232812504
Principal point of view; x: 3.198308916796875, y: 2.3934605770833333
Aspect ratio: 1.0005929569269807

GoPro specifications

https://gopro.com/help/articles/Question_Answer/HERO4-Field-of-View-FOV-Information

Edit

Matlab calibration results

Focal Length:          fc = [ 582.18394   582.52915 ] ± [ 0.77471   0.78080 ]
Principal point:       cc = [ 663.50655   378.74541 ] ± [ 1.40781   1.13965 ]
Skew:             alpha_c = [ -0.00028 ] ± [ 0.00056  ]   => angle of pixel axes = 90.01599 ± 0.03208 degrees
Distortion:            kc = [ -0.25722   0.09022   -0.00060   0.00009  -0.01662 ] ± [ 0.00228   0.00276   0.00020   0.00018  0.00098 ]
Pixel error:          err = [ 0.30001   0.28188 ]

One of the images used for calibration

And the undistorted image

Dichotomous

Answer 1

You have entered 6.17mm and 4.55mm for the sensor size in OpenCV, which corresponds to an aspect ratio 1.36 whereas as your resolution (1270x720) is 1.76 (approximately 16x9 format). Did you crop your image before MATLAB calibration?

The pixel size seems to be 1.55µm from this Gopro page (this is by the way astonishingly small!). If pixels are squared, and they should be on this type of commercial cameras, that means your inputs are not coherent. Computed sensor size should be :

[Sensor width, Sensor height] = [1280, 720]*1.55*10^-3 = [1.97, 1.12] mm

Even if considering the maximal video resolution which is 3840 x 2160, we obtain [5.95, 3.35]mm, still different from your input.

Please see this explanation about equivalent focal length to understand why the actual focal length of the camera is not 17.2 but 17.2*5.95/36 ~ 2.8mm. In that case, compute FOV using the formulas here for instance. You will indeed find values of 93.5°/61.7° (close to your outputs but still not what is written in the specifications because there probably some optical distortion due to the wide angles).

What I do not understand though, is how the focal distance returned can be right whereas sensor size entered is wrong. Could you give more info and/or send an image?

Edits after question updates

On that cameras, with a working resolution of 1280x720, the image is downsampled but not cropped so what I said above about sensor dimensions does not apply. The sensor size to consider is indeed the one used (6.17x4.55) as explained in your first comment.

The FOV is constrained by the calibration matrix inputs (fx, fy, cx, cy) given in pixels and the resolution. You can check it by typing:

2*DEGRES(ATAN(1280/(2*582.18394))) (=95.416776...°)

This FOV value is smaller than what is expected, but by the look of the undistorted image, your MATLAB distortion model is right and the calibration is correct. The barrel distortion due to the wide angle seems well corrected by the the rewarp you applied.

However, MATLAB toolbox uses a pinhole model, which is linear and cannot account for intrinsic parameters such as lens distortion. I assume this from the page :

https://fr.mathworks.com/help/vision/ug/camera-calibration.html

Hence, my best guess is that unless you find a model which fits more accurately the Gopro camera (maybe a wide-angle lens model), MATLAB calibration will return an intrinsic camera matrix corresponding to the "linear" undistorted image and the FOV will indeed be smaller (in the case of barrel distortion). You will have to apply distortion coefficients associated to the calibration to retrieve the actual FOV value.

We can see in the corrected image that side parts of the FOV get rejected out of bounds. If you had warped the image entirely, you would find that some undistorted pixels coordinates exceed [-1280/2;+1280/2] (horizontally, and idem vertically). Then, replacing opencv.core.Size(1280, 720) by the most extreme ranges obtained, you would hopefully retrieve Gopro website values.

In conclusion, I think you can rely on the focal distance value that you obtained if you make measurements in the center of your image, otherwise there is too much distortion and it doesn't apply.

Preinstruct

Thanks a lot for your answer. I found the sensor's dimensions here vfxcamdb.com/category/camera/gopro. Regardless, the field of view returned by calibrationMatrixValues(...) does not change if I alter the sensor's dimensions. I presume all the data necessary for this calculation is taken from the camera's calibration matrix. What changes, however, is the focal length which then equals to ~0.9mm (when I change the sensor's dimensions to 1.97 and 1.12). – Dichotomous

I have included some more data in the post above. – Dichotomous

I had not found the info about sensor size but you should indeed rely on Gopro infos when you have them. My mistake then: I guess there is some spacing between pixels and that the downsampling at lower resolutions virtually increases pixel pitch. Results about focal make sense to me now. About the FOV discrepancies, have you tried using a distortion model adapted to fisheye lenses for calibration? – Preinstruct

I have just tried using the fisheye model for camera calibration and I got an almost identical camera matrix while only the fisheye distortion coefficients differ and using them for image undistortion does not yield good results. As for camera calibration I used 19 images. I am not sure what to do next. Maybe I just misinterpreted the resulting FOVs? – Dichotomous

Ok. Just to be sure, the material on which you put your chessboard is rigid? What we could do to help us understand is measure the field of view manually. just put the chessboard patch at a known distance from the camera (not too close, so that the focus is okay, and on a plane orthogonal to the camera axis (optical axis). It has to be facing the camera quite well. Ideally, use a larger thing than a chessboard so the plate takes all the image. Then, calculate the angle that you see, using basic geometry or send me the picture + dimensions of the plates and distance from camera and I'll do it. – Preinstruct

I have done the experiment you have proposed and the FOVs are indeed as the ones in GoPro's specs ~(118/69). Therefore I conclude there are two possible explanations. 1) The camera calibration is incorrect. What I would not understand is why image undistortion and calculating the the camera's pose using this very same matrix seems to be working quite well. 2) The resulting camera matrix corresponds to the undistorted image. I have tried to do the same experiment using undistorted images and the resulting FOVs were ~(96,64) and I am sure I inevitably made some slight measurement errors. – Dichotomous

Oh and yes, the material is rigid and flat. – Dichotomous

OK, then I agree with your #2 explanation. I did an additional edit to my answer accordingly, please feel free to modify or close the question. – Preinstruct

Yes, it all makes sense now. Thank you tenfold for your explanation and your time! – Dichotomous