OpenCV to OpenGL coordinate system transform

Question

I have two right handed coordinate systems.

OpenCV

As you can see with the black arrows, the camera looks down the positive $Z$ axis. You can ignore the rest of the diagram.

OpenGL

Although not visible here, the camera in OpenGL looks down the -Z axis. I want to transform a 3D point in front of the camera in the OpenCV coordinate system to a 3D point in front of the camera in the OpenGL.

I'm trying to represent this in a 4x4 matrix that concatenates R and T with 0001 at the bottom.

So far, I've tried this

1  0  0  0
0 -1  0  0
0  0  -1 0
0  0  0  1

but it doesn't seem to do anything, nothing shows up in the OpenGL coordinate system.

Busy

By your diagrams, your answer is the correct one, it should change the sign of the Y and Z components of a 3D point... can you elaborate on the do nothing part? – Bentham

The point still seems to be rendering behind the OpenGL camera, rather than the Z component being flipped – Busy

What is the source of the input points? could it be that is not in OpenCV coordinate system? to check if your transformation is at least doing what is suppose to, try to apply it to unit vectors (e.g. [0,1,0]) to see if they get rotated. Another way to double check this is using OpenCV viz to debug/visualize the points before and after the transformation – Bentham

@Bentham the provided screenshot is taken from OpenCV's website. I'm using solvePnP to compute the rotation and translation vector from some "object points" describing a 3D object and "image points" which is where it appears on the screen. Plotting unit vectors seems like a good idea – Busy

Remember that additionally to your axis transformations the matrices in opencv and opengl are different: the first one are row major order, the second one are column major order. That could be a source of problem if you're dealing with transformation matrices – Laliberte

Yeah I transpose the final matrix before giving it to OpenGL – Busy

What are you using for your projection matrix? can you post a snippet of the code showing how you use/convert OpenCV parameters to the OpenGL one. – Obrian

Answer 1

The camera coordinates of OpenCV goes X right, Y down, Z forward. While the camera coordinates of OpenGL goes X right, Y up, Z inward.

Use solvePnP as one of the most commonly used example. You get a 3x3 rotation matrix R and a 1x3 translation vector T, and create a 4x4 view matrix M with R and T. Simply inverse the 2nd and 3rd row of M and you will get a view matrix for OpenGL rendering.

Gemination

This is a great answer and helped me but it lacks the fact that OpenGL uses a different notation for it's matrices compared to OpenCV. Since OpenGL uses a column-major system, it will read the the first column as if it was the first row. Because of this, we need to transpose the "OpenGL view matrix" before we use it. – Despond

@Despond thanks for the supplements, row-column conversion is an important point among matrices. – Gemination

How do I convert OpenGL camera coordinates to OpenCV? I've OpenGL coordinates obtained from ARCore camera poses, I want it to be converted to OpenCV format. – Donato