OpenCV Equirectangular Rotation

Question

I'm currently stuck on achieving an equirectangular rotation on a 360° image with OpenCV because of my mathematical understanding (nearly zero) of projections and rotations matrixes.

The result of a such rotation would be exactly what you can see here: https://www.youtube.com/watch?v=l1N0lEKIeLA

I found some code here: https://github.com/FoxelSA/libgnomonic/wiki/Equirectangular-rotation_v0.1 but I didn't succeed to apply it to opencv

If someone has any idea how to apply it for an OpenCV Mat and Pitch, Yaw, Roll angles it would be highly appreciated!

Thanks!

Chrysostom

What has been done so far? (This isn't a request for someone to do the coding for you, is it?) – Cicatrize

This is a tricky problem. I would think it's a little similar to asking how to filter colors in OpenCV (multiple solutions, but there are built in functions). I'm hoping someone can point to some functionality of OpenCV that I've overlooked that does this already or how to adapt OpenCV to perform this task. – Adulterant

Answer 1

Instead of talking about yaw, pitch and roll, I'll talk here about Euler angles x, y and z.

To perform a rotation of your equirectangular mapping, you can follow this procedure:

• Consider coordinates (i2, j2) in your result image. We'll try to find which color to put here. These coordinates correspond to a point on the sphere with latitude lat2 = 180 * i2 / image.height and longitude lon2 = 360 * j2 / image.width. Compute the corresponding 3D vector v2.

• Compute the rotation matrix R with angles x, y and z (look at the formulas here). Take the transpose of this matrix to get the inverse rotation from the new image to the old one. We'll name this inverse rotation matrix Rt.

• Compute v1 = Rt * v2. Then compute the latitude lat1 and longitude lon1 of v1.

• Find the color in the original image at coordinates i1 = image.height * lat1 / 180 and j1 = image.width * lon1 / 360. This might not be integer coordinates. You might have to interpolate between several pixels to get your value. This is the color of the pixel at position (i2, j2) in your new image.

You'll need to look at how to convert between 3D vectors on a sphere and their latitude and longitude angles but this shouldn't be too hard to find. The algorithm described here should be rather straightforward to implement.

Let me know if I made any mistake as I haven't tested it myself.

Strangles

This is good but I mostly placed the bounty in the hopes that OpenCV had functions specifically intended to perform some (or all) of these steps. The application of a rotation matrix to the original image to get the rotated image seems like a (complex) operation that might be available in OpenCV already, but I haven't found anything in the docs thus far. That being said, thank you for outlining the overall process and coordinate transforms involved. If there aren't any OpenCV algorithms, this answer should be excellent for writing my own. – Adulterant

@TaviKohn I don't think you'll find ready solutions in OpenCV. However, it does have some tools that could make implementing your own algorithm more convenient: matrix product is already there and the remap function lets you avoid the tedious task of interpolating the pixels by yourself. – Strangles

Excellent! Thank you very much for your help. – Adulterant