Is there a fisheye or dual fisheye to equirectangular filter for ffmpeg?

Question

Or a way to do it with an existing filter? So that you could take in video from a fisheye or dual fisheye camera (such as the Ricoh Theta) and directly output equirectangular, in real-time, to something like RTMP?

Ferebee

Was you able to accomplish that? – Iolite

Yes see the accepted answer below, it works! – Ferebee

Answer 1

The Remap filter does just this:

This filter copies pixel by pixel a source frame to a target frame. It remaps the pixels to a new x,y destination based on two files ymap/xmap.

Basic command syntax is

ffmpeg -i fisheye_grid_input.jpg -i fisheye_grid_xmap.pgm -i fisheye_grid_ymap.pgm -filter_complex remap out.png 

Also included at that link are the mapping files for

Ricoh Theta S camera: input files for resolution 1920x960(1080)

Wearproof

Thank you for reply. Do you know where can I find PGM files for Ricoh Theta S 1280x720 resolution? – Froude

Is there a way to do this for an existing video? instead of just an image? – Teheran

Sure. Replace first input with video. Insert -loop 1 before 2nd and 3rd inputs. And output to video. – Wearproof

@Froude I've extended the projection tool so you can generate projection files for dual-fisheye cameras too, you can get it at github.com/raboof/dualfisheye2equirectangular – Orangy

How can I tune the maps to make them compatible with different FOVs? My camera has 235° FOV, others have 220 or 190 or 180. A FOV >180 is mandatory to have a good stitching, but SW must take into account the overlapping. – Nertie

Excelent, but do you know how to apply the remap filter over a camera output instead of a single image? – Iolite

Use a video as the first input. If the filter doesn't reuse the X/Ymap images, add -loop 1 before each of the image inputs. – Wearproof

It worked with videos and webcam streams but, when I try to get this work with ffserver and a webcam, I have the following error "MPEG 1/2 does not support 15/1 fps". I dont know how to make it work – Iolite

Add -r 30 to change output to 30 fps. – Wearproof

Answer 2

In the latest ffmpeg, you can do this to convert fisheye video to equirectangular now

ffmpeg -y -i in.mp4 -vf v360=dfisheye:e:yaw=-90 -c:v libx265 -b:v 40000k -bufsize 5000k -preset ultrafast -c:a copy out.mp4

• y : overwrite output without wanring
• i xxx : input file
• vf yyy: use filter

  • yyy: filter parameters

  • v360 : filter name

    • dfisheye : double fisheye (rectangular image containing two spheres/fisheye); use "fisheye" to use single sphere/fisheye
    • e : abbreviation for "equirectangular"
    • yaw : view direction (=azimut) of center of equirectangular output (=look left/right); use "pitch" to look up/down
    • ih_fov : input horizontal Field Of View; half sphere is 180°, but some cameras arrive to 235°
    • iv_fov : input vertical Field Of View, usually identical to ih_fov
    • h_fov : output horizontal FOV
    • v_fov : output vertical FOV

Docs: https://ffmpeg.org/ffmpeg-filters.html#v360

Note: filter works fine both with image or video as input

Wolver

some comments to the parameters would be useful; I only know the ones for images... – Nertie