BeagleBone Black OpenCV Python is too slow

Question

I try to get images from webcam wtih opencv and python. Code is so basic like:

import cv2
import time
cap=cv2.VideoCapture(0)
cap.set(cv2.cv.CV_CAP_PROP_FRAME_WIDTH,640)
cap.set(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT,480)
cap.set(cv2.cv.CV_CAP_PROP_FPS, 20)

a=30
t=time.time()
while (a>0):
        now=time.time()
        print now-t
        t=now
        ret,frame=cap.read()
        #Some processes
        print a,ret
        print frame.shape
        a=a-1
        k=cv2.waitKey(20)
        if k==27:
                break
cv2.destroyAllWindows()

But it works slowly. output of program:

VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
VIDIOC_QUERYMENU: Invalid argument
HIGHGUI ERROR: V4L: Property <unknown property string>(5) not supported by device
8.82148742676e-06
select timeout
30 True
(480, 640, 3)
2.10035800934
select timeout
29 True
(480, 640, 3)
2.06729602814
select timeout
28 True
(480, 640, 3)
2.07144904137
select timeout

Configuration:

• Beaglebone Black RevC
• Debian-wheezly
• opencv 2.4
• python 2.7

Dickey

For this problem it probably won't help, but if you're using opencv to actually do cv stuff, you probably want to check that the opencv you're using is compiled against the NEON extensions. (doing google opencv beaglebone neon will probably find useful details, e.g. blog.lemoneerlabs.com/3rdParty/Darling_BBB_30fps_DRAFT.html) – Banlieue

"opencv 2.4" -- you need to be more specific, there has been a number of 2.4.x releases of OpenCV over the course of several years. – Brahui

Answer 1

I encountered a similar problem when I was working on a project using OpenCV 2.4.9 on the Intel Edison platform. Before doing any processing, it was taking roughly 80ms just to perform the frame grab. It turns out that OpenCV's camera capture logic for Linux doesn't seem to be implemented properly, at least in the 2.4.9 release. The underlying driver only uses one buffer, so it's not possible to use multi-threading in the application layer to work around it - until you attempt to grab the next frame, the only buffer in the V4L2 driver is locked.

The solution is to not use OpenCV's VideoCapture class. Maybe it was fixed to use a sensible number of buffers at some point, but as of 2.4.9, it wasn't. In fact, if you look at this article by the same author as the link provided by @Nickil Maveli, you'll find that as soon as he provides suggestions for improving the FPS on a Raspberry Pi, he stops using OpenCV's VideoCapture. I don't believe that is a coincidence.

Here's my post about it on the Intel Edison forum: https://communities.intel.com/thread/58544.

I basically wound up writing my own class to handle the frame grabs, directly using V4L2. That way you can provide a circular list of buffers and allow the frame grabbing and application logic to be properly decoupled. That was done in C++ though, for a C++ application. Assuming the above link delivers on its promises, that might be a far easier approach. I'm not sure whether it would work on BeagleBone, but maybe there's something similar to PiCamera out there. Good luck.

EDIT: I took a look at the source code for 2.4.11 of OpenCV. It looks like they now default to using 4 buffers, but you must be using V4L2 to take advantage of that. If you look closely at your error message HIGHGUI ERROR: V4L: Property..., you see that it references V4L, not V4L2. That means that the build of OpenCV you're using is falling back on the old V4L driver. In addition to the singular buffer causing performance issues, you're using an ancient driver that probably has many limitations and performance problems of its own.

Your best bet would be to build OpenCV yourself to make sure that it uses V4L2. If I recall correctly, the OpenCV configuration process checks whether the V4L2 drivers are installed on the machine and builds it accordingly, so you'll want to make sure that V4L2 and any related dev packages are installed on the machine you use to build OpenCV.

Ucayali

the buffer is in the hardware memory i think (from my experience). – Artina

Answer 2

The "secret" to obtaining higher FPS when processing video streams with OpenCV is to move the I/O (i.e., the reading of frames from the camera sensor) to a separate thread.

When calling read() method along with cv2.VideoCapture function, it makes the entire process very slow as it has to wait for each I/O operation to be completed for it to move on to the next one (Blocking Process).

In order to accomplish this FPS increase/latency decrease, our goal is to move the reading of frames from a webcam or USB device to an entirely different thread, totally separate from our main Python script.

This will allow frames to be read continuously from the I/O thread, all while our root thread processes the current frame. Once the root thread has finished processing its frame, it simply needs to grab the current frame from the I/O thread. This is accomplished without having to wait for blocking I/O operations.

You can read Increasing webcam FPS with Python and OpenCV to know the steps in implementing threads.

---

EDIT

Based on the discussions in our comments, I feel you could rewrite the code as follows:

import cv2

cv2.namedWindow("output")
cap = cv2.VideoCapture(0)

if cap.isOpened():              # Getting the first frame
    ret, frame = cap.read()
else:
    ret = False

while ret:
    cv2.imshow("output", frame)
    ret, frame = cap.read()
    key = cv2.waitKey(20)
    if key == 27:                    # exit on Escape key
        break
cv2.destroyWindow("output")

Flump

I did same things but python and opencv still too slow – Dickey

Try, setting your camera fps to cap.set(cv2.cv.CV_CAP_PROP_FPS, 30) and see the results. – Flump

Your best bet now would be to compile OpenCV without installing the development package for V4L. – Flump

Answer 3

try this one ! I replaced some code in the cap.set() section

import cv2
import time
cap=cv2.VideoCapture(0)
cap.set(3,640)
cap.set(4,480)
cap.set(5, 20)

a=30
t=time.time()
while (a>0):
        now=time.time()
        print now-t
        t=now
        ret,frame=cap.read()
        #Some processes
        print a,ret
        print frame.shape
        a=a-1
        k=cv2.waitKey(20)
        if k==27:
                break
cv2.destroyAllWindows()

output (pc webcam) your code was wrong for me.

>>0.0
>>30 True
>>(480, 640, 3)
>>0.246999979019
>>29 True
>>(480, 640, 3)
>>0.0249998569489
>>28 True
>>(480, 640, 3)
>>0.0280001163483
>>27 True
>>(480, 640, 3)
>>0.0320000648499

Artina

The original cap.set function calls were fine. The constants @Dickey used evaluate to the same values you've manually specified. That's not the cause of the performance problem. – Ucayali