Making a video with opencv and ffmpeg. How to find the right color format?
Asked Answered
P

6

10

I have a webcam video recorder program built with python, opencv and ffmpeg

It works ok except that the color of the video is more blue than the reality. The problem seems to come from color format of images.

It seems that OpenCv is giving BGR images and ffmpeg+libx264 is expecting YUV420p. I've read that YUV420p correspond to YCbCr.

opencv has no conversion from BGR to YCbCr. It only has a conversion to YCrCb.

I have made some searchs and tried different alternatives to try converting opencv image to something that could be ok for ffmpeg+libx264. None is working. At this point, I am a bit lost and I would appreciate any pointer that could help me to fix this color issue.

Priscilapriscilla answered 19/9, 2012 at 21:44 Comment(0)
C
13

You are right, the default pixel format of OpenCV is BGR.

The equivalent format on the ffmpeg side would be BGR24, so you don't need to convert it to YUV420p if you don't want to.

This post shows how to use a python application to capture frames from the webcam and write the frames to stdout. The purpose is to invoke this app on the cmd-line and pipe the result directly to the ffmpeg application, which stores the frames on the disk. Quite clever indeed!

capture.py:

import cv, sys

cap = cv.CaptureFromCAM(0)
if not cap:
    sys.stdout.write("failed CaptureFromCAM")

while True :
    if not cv.GrabFrame(cap) : 
        break

    frame = cv.RetrieveFrame(cap)
    sys.stdout.write( frame.tostring() )

And the command to be executed on the shell is:

python capture.py | ffmpeg -f rawvideo -pix_fmt bgr24 -s 640x480 -r 30 -i - -an -f avi -r 30 foo.avi

Where:

  • -r gives the frame rate coming off the camera
  • -an says "don't encode audio"

I tested this solution on my Mac OS X with OpenCV 2.4.2.

EDIT:

In case you haven't tried to record from the camera and use OpenCV to write the video to an mp4 file on the disk, here we go:

import cv, sys

cap = cv.CaptureFromCAM(0)   # 0 is for /dev/video0
if not cap:
    sys.stdout.write("!!! Failed CaptureFromCAM")
    sys.exit(1)

frame = cv.RetrieveFrame(cap)
if not frame: 
    sys.stdout.write("!!! Failed to retrieve first frame")
    sys.exit(1)

# Unfortunately, the following instruction returns 0
#fps = cv.GetCaptureProperty(cap, cv.CV_CAP_PROP_FPS)
fps = 25.0      # so we need to hardcode the FPS
print "Recording at: ", fps, " fps"  

frame_size = cv.GetSize(frame)
print "Video size: ", frame_size  

writer = cv.CreateVideoWriter("out.mp4", cv.CV_FOURCC('F', 'M', 'P', '4'), fps, frame_size, True)
if not writer:
    sys.stdout.write("!!! Error in creating video writer")
    sys.exit(1)


while True :
    if not cv.GrabFrame(cap) : 
        break
    frame = cv.RetrieveFrame(cap)
    cv.WriteFrame(writer, frame)

cv.ReleaseVideoWriter(writer)
cv.ReleaseCapture(cap)

I've tested this with Python 2.7 on Mac OS X and OpenCV 2.4.2.

Cornu answered 24/9, 2012 at 21:43 Comment(8)
Thanks for your answer. I didn't have time to check the solution yet but it looks interesting. I've tried to use the bgr24 pix_fmt already but it doesn't work with the libx264 codec. I am not sure if I really need to use libx264. I need a cross-platform solution (windows, mac, linux) and as a consequence libx264 may be required. I need to check.Priscilapriscilla
On Windows libx264 is the default codec and cause the image to have wrong colors because it is not compatible with bgr24 pix_fmtPriscilapriscilla
Sorry to ask, but why don't you use OpenCV to create the video file?Cornu
Don't OpenCV use ffmpeg for creating the video? Would be happy to get more info about an alternative solution with OpenCV.Priscilapriscilla
Thanks for the code but it doesn't work: ReleaseVideoWriter is missing. I tried to del the writer but it doesn't work as well. See https://mcmap.net/q/911280/-unable-to-create-a-basic-video-file-using-opencv/117092Priscilapriscilla
Sorry to hear that, but the code was tested and it does work on the system I mentioned. Thanks for the link, I've forgotten I answered that question too :)Cornu
I think the pb is relative to windows platform. Anyway, thank for you answer which is very interesting. It didn't solve my problem but it (and all the impressive number of questions you answered on opencv) helped me to understand better how opencv is working and to solve my issue. It worths the bounty :)Priscilapriscilla
I read again your answer and in fact it was the right answer from the very beginning. It was just my implementation which was not correct. It worths to be accepted too :)Priscilapriscilla
C
1

Have you tried switching the Cb/Cr channels in OpenCV using split and merge ?

Care answered 29/9, 2012 at 22:27 Comment(3)
I tried different conversions. Which one would you recommend?Priscilapriscilla
I suggest you just swap the Cb and Cr channels. In YCbCr, Y stands for the "Luminance" part, that is more or less the gray level. The color informations are stored in 2 "Chrominance" channels, Cr (as in Chrominance Red) and Cb (chrominance blue). As you said, OpenCV converts to YCrCb, and ffmpeg YUV = YCbCr. This + the fact that your images appear blueish makes me think that you could simply swap the chrominance channels, Cr<->Cb.Care
Thanks. It helped to find the solution. I've made a converter using Split and Merge and I realized that I had a double-conversion and that using a Copy was enough :)Priscilapriscilla
K
0

Checked the conversion formulas present in: http://en.wikipedia.org/wiki/YCbCr?

Katmandu answered 19/9, 2012 at 22:48 Comment(1)
Thanks for the link but I am a bit lost between all these formulas. What would you recommend in order to implement it?Priscilapriscilla
P
0

The libx264 codec is able to process BGR images. No need to use any conversion to YCbCr. NO need to give a spcific pix_ftm to ffmpeg. I was using RGB and it was causing the blueish effect on the video.

The solution was simply to use the original image retuned by the camera without any conversion. :)

I tried this in my previous investigation and it was crashing the app. The solution is to copy the frame returned by the camera.

    frame = opencv.QueryFrame(camera)
    if not frame:
        return None, None

    # RGB : use this one for displaying on the screen
    im_rgb = opencv.CreateImage(self.size,  opencv.IPL_DEPTH_8U, 3)
    opencv.CvtColor(frame, im_rgb, opencv.CV_BGR2RGB)

    # BGR : Use this one for the video
    im_bgr = opencv.CreateImage(self.size,  opencv.IPL_DEPTH_8U, 3)
    opencv.Copy(frame, im_bgr)

    return im_rgb, im_bgr
Priscilapriscilla answered 5/10, 2012 at 9:36 Comment(0)
G
0

I've already answered this here. But my VidGear Python Library automates the whole process of pipelining OpenCV frames into FFmpeg and also robustly handles the format conversion. Here's a basic python example:

# import libraries
from vidgear.gears import WriteGear
import cv2

output_params = {"-vcodec":"libx264", "-crf": 0, "-preset": "fast"} #define (Codec,CRF,preset) FFmpeg tweak parameters for writer

stream = cv2.VideoCapture(0) #Open live webcam video stream on first index(i.e. 0) device

writer = WriteGear(output_filename = 'Output.mp4', compression_mode = True, logging = True, **output_params) #Define writer with output filename 'Output.mp4' 

# infinite loop
while True:

    (grabbed, frame) = stream.read()
    # read frames

    # check if frame empty
    if not is grabbed:
        #if True break the infinite loop
        break


    # {do something with frame here}
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # write a modified frame to writer
        writer.write(gray) 

        # Show output window
    cv2.imshow("Output Frame", frame)

    key = cv2.waitKey(1) & 0xFF
    # check for 'q' key-press
    if key == ord("q"):
        #if 'q' key-pressed break out
        break

cv2.destroyAllWindows()
# close output window

stream.release()
# safely close video stream
writer.close()
# safely close writer

Source: https://github.com/abhiTronix/vidgear/wiki/Compression-Mode:-FFmpeg#2-writegear-classcompression-mode-with-opencv-directly

You can check out full VidGear Docs for more advanced applications and exciting features.

Hope that helps!

Gearbox answered 1/5, 2019 at 14:5 Comment(0)
C
0

Getting frames a video using ffmpeg and opencv gave me different colors. This line of code solved the problem:

frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

Result:

Result

Curfew answered 16/5 at 22:13 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.