Is there a way to remove GoPro fisheye using ffmpeg

Question

It seems possible to compensate the fisheye distortion using ffmpeg thanks to the lenscorrection filter:

ffmpeg -i in.mp4 -vf "lenscorrection=cx=0.5:cy=0.5:k1=-.25:k2=-.25" out.mp4

What values for k1 and k2, particularly for a gopro? Is there a way to compute them?

Italia

Duplicate over in video: video.stackexchange.com/questions/12667/… – Summertree

@abernier I think providing an example input image (video grab) would speed up getting some Answers. – Drumlin

Answer 1

I don't think there's a generic one setting(s) to cover all GoPro models & lenses. Factor in also that people can use some customised/replacement lenses.

I was hoping for an example image featuring what you have to work with, but never mind...
Hope the below is useful to you in some way

Check this link for GoPro Hero 3 settings (applies to other models). To get a rough idea of best k1 and k2 values, you should find the nearest aspect ratio to your image resolution then...

Divide the H. FOV deg with the V. FOV deg and multiply result with Diag. FOV deg.

Example: where picture size is 1920 x 1080 : This makes it 16:9 widescreen.

GoPro 3 setting: 16 x 9 Widescreen :: H fov = 69.5 || V fov = 118.2 || Diag = 133.6

So 118.2 / 69.5 * 133.6 = 227.216 etc

for k1 the result is now integer 227 (drop the decimal points).

for k2 the value 0.022 seems universal (all test images), so you can try k2=-.022.

for cx and cy keep them as : cx=0.5:cy=0.5:.

Example:

ffmpeg -i in.mp4 -vf "lenscorrection=cx=0.5:cy=0.5:k1=-0.227:k2=-0.022" out.mp4

Result : (original image found via Google images of a GoPro Hero-2 video).

Drumlin

Excellent! You've just won the bounty ;) However, can you explain the k2 value a bit more? Why it is "universal" ? Thanks again – Evenhanded

@Evenhanded Thanks. About k2 it's not so much that it's universal but just that it works for that type of fish-eye lens. My test pictures all had an "image inside a circle" type of look (with black edges). It may not be best for panoramic lens (distorted image but fils screen without black edges). I'm off on a mission right now, but will update with extra info/examples later tonight. – Drumlin

Answer 2

For GoPro Hero 5 and 6:

ffmpeg -i input.mp4 \
    -vf 'lenscorrection=k2=0.006:k1=-0.18' \
    output.mp4

source

Injure

Answer 3

FFmpeg has a lenscorrection filter that can be used to correct radial distortion in videos. That said, without more context - such as the model and settings of your camera - it would be hard to provide more information. I suspect it would require a bit of trial-and-error anyway.

PS: Have you tried running the camera's software under WINE? Some versions of it apparently have this kind of correction filter predefined and easily accessible.

Zest

Thanks for the answer, but I am looking for ffmpeg parameters for lenscorrection (or other opensource, without using 3rd party software). – Italia

yep, thx for the answer but specific values for k1 and k2 would be awesome for gopro – Evenhanded

Answer 4

I wrote a post about various methods for correcting distortion using ffmpeg - I think the lensfun or v360 filters are better options than the lenscorrection filter.

That said, in theory you should be able to calculate values for k1 and k2 using OpenCV- calibrateCamera function. You would want to use the flags CALIB_FIX_PRINCIPAL_POINT (along with coordinates to fix the principal point at the center of the image as lenscorrection assumes), CALIB_FIX_K3 (since lenscorrection does not use k3), and CALIB_FIX_S1_S2_S3_S4 (since lenscorrection does not consider tangential distortion). There is example code available that you can simply modify to do this.

However when I tried, I got stuck because I couldn't find a way to fix the focal length in OpenCV, but in lenscorrection the focal length is hard coded to half of the image diagonal.

Conductance

Answer 5

For GoPro 3+, 16:9 with FOV Wide

ffmpeg -i input.mp4 -vf "lenscorrection=0.5:0.5:-0.335:0.097" output.mp4

Endomorphism

Answer 6

You can fix fisheye using ffmpeg's lenscorrection, but the result will look bad on edges.

Woald