Reading Frame Buffer from camera using ctypes

Question

I'm having problems to read out the frame buffer of a camera using python. The camera (Xenics) is connected via USB and there is a dll which ships with the camera. I access this dll using ctypes. The code I am using is mainly inspired by a python module for lantz (lantz_drivers_xenics).
The dll provides the function XC_CopyFrameBuffer which I am using to copy the framebuffer to a numpy array.

While in principle the acquisition works fine, the trouble is to continuously read the framebuffer at moderate or high speed. The compiled software that ships with the camera is able to read out the camera at 24 fps. In my program I would like to read the framebuffer at something between 5 and 25 fps. However, I cannot reach this value, because the time it takes to call the XC_CopyFrameBuffer function varies between 0 and ~0.5 seconds; it is mostly low, but 0.5 seconds for roughly every third frame or so.

My real program is more complex, uses threads and processes the acqired images. But I have boiled down the code to a minimal example that reproduces the problem and that I have attached. It's text and graphical output are shown below. My question is:

1. Has anyone experienced similar issues with cameras?
2. Has anyone used a Xenics camera, connected to it via ctypes in python and could provide a working example of that?
3. Is there some apparent or hidden problem in the way I (and/or the lantz_drivers_xenics) uses the dll?

This is the code I am using:

import time
import ctypes 
import threading
import numpy as np
from numpy.ctypeslib import ndpointer


calibration_file =  r"C:\Path\to\some\calibrationfile.xca"
dll = "C:\\Programme\\X-Control\\xcamera.dll"
lib = ctypes.WinDLL(dll) #for xcamera we need ctypes.WinDLL

exposure_time = 300 #microseconds (us)
readlock = threading.Lock()

#add types    
lib.XC_OpenCamera.argtypes = [ctypes.c_uint]
lib.XC_CloseCamera.argtypes = [ctypes.c_uint]
lib.XC_IsInitialised.argtypes = [ctypes.c_uint]
lib.XC_LoadFromFile.argtypes = [ctypes.c_uint, ctypes.c_char_p]
lib.XC_LoadCalibrationPack.argtypes = [ctypes.c_uint, ctypes.c_char_p]
lib.XC_SetGainCamera.argtypes = [ctypes.c_uint, ctypes.c_double]
lib.XC_SetIntegrationTime.argtypes = [ctypes.c_uint, ctypes.c_ulong]

lib.XC_SetFan.argtypes = [ctypes.c_uint, ctypes.c_bool]
lib.XC_StartCapture.argtypes = [ctypes.c_uint]
lib.XC_StopCapture.argtypes = [ctypes.c_uint]
lib.XC_GetFrameSizeInBytes.argtypes = [ctypes.c_uint]
lib.XC_CloseCamera.restype = ctypes.c_void_p
lib.XC_StopCapture.restype = ctypes.c_void_p

xcamera_id = lib.XC_OpenCamera(0)

#lib.XC_LoadFromFile(xcamera_id,  self.config_file)
lib.XC_LoadCalibrationPack(xcamera_id,  calibration_file)
height = lib.XC_GetHeight(xcamera_id)
width = lib.XC_GetWidth(xcamera_id)
shape = (height, width)
lib.XC_CopyFrameBuffer.argtypes = [ctypes.c_uint, ndpointer(dtype=np.uint16, shape=shape),ctypes.c_uint]
frame_size = lib.XC_GetFrameSizeInBytes(xcamera_id)
fbuffer = np.zeros(shape=shape, dtype=np.uint16)

lib.XC_SetIntegrationTime(xcamera_id, exposure_time)
lib.XC_SetFan(xcamera_id,  True)

lib.XC_StartCapture(xcamera_id)

times = []

for i in range(150):

    time.sleep( exposure_time/1000000. ) #  1./25
    if readlock.acquire() : 
        start_time = time.time()
        (lib.XC_CopyFrameBuffer(xcamera_id, fbuffer, frame_size) )
        #time.sleep(0.0002)
        now = time.time()
        readlock.release()

        print "Frame {nr}\ttime: {time}".format(time=now-start_time, nr=i)
        times.append(now-start_time)
    else:
        time.sleep(0.001)
        print "Try again" #This gets never printed, so readlock works fine.


###### CLOSING ###########
lib.XC_StopCapture(xcamera_id)
lib.XC_SetFan(xcamera_id,  False)
lib.XC_CloseCamera(xcamera_id)                                         

###### Plot ###########
import matplotlib.pyplot as plt
fig = plt.figure()
ax = fig.add_subplot(121)
ax.imshow(fbuffer, interpolation="none")
ax2 = fig.add_subplot(122)
ax2.plot(times)
plt.savefig("testing/readout.png")
plt.show()

A typical output looks something like this, where time is the time it takes to copy the frame buffer.

Frame 72    time: 0.0
Frame 73    time: 0.0
Frame 74    time: 0.000999927520752
Frame 75    time: 0.512000083923
Frame 76    time: 0.000999927520752
Frame 77    time: 0.0
Frame 78    time: 0.516000032425
Frame 79    time: 0.0
Frame 80    time: 0.000999927520752
Frame 81    time: 0.516000032425
Frame 82    time: 0.0
Frame 83    time: 0.0
Frame 84    time: 0.514000177383
Frame 85    time: 0.0
Frame 86    time: 0.0759999752045

And the graph plotted is this (where the image looks exactly like it's supposed to). The right graph shows the time in seconds for each frame

Comments:

• I am working in python 2.7
• I played around with some of the parameters:
  • Adding time.sleep(x) with different values for x at different positions in the loop has the effect of increasing the number of frames which take 0.5s to read. The longer the sleep time, the more long frame reads.
  • Ommiting time.sleep( exposure_time/1000000. ) provides empty frames.
  • The problem is independent of whether I use readlock.acquire() or not.
  • The problem is independent of the number of loops.

Kerman

I took a quick look at the specs of what I think is your camera (Xeva-1.7-640), and it sounds like it may be UVC (USB Video Class, the standard for USB cameras) compliant. In this case, you should be able to bypass the dll and directly use, e.g., OpenCV to capture from the camera (sample code), which is commonly used with UVC-compliant webcams and hence likely has less issues. – Mcmasters

@Ulrich Stern It would indeed be great to be able to use openCV directly. However openCV does not really seem to support all of the camera's settings, see e.g. here . Especially I have not been able to load the calibration file to the camera. If anyone has successfully used a Xenics camera with openCV, I'd be happy to hear about it. – Kerman

You could possibly use the dll for settings and OpenCV to capture. In one of my projects, I used v4l2-ctl (from Python on Linux) to change focal plane, exposure, etc. of Microsoft webcams and then OpenCV to capture. (If you hook your camera up to a Linux box, you can list all UVC controls (settings) via v4l2-ctl -d 0 -l. My hunch is you would still need the dll for the calibration.) – Mcmasters

Does the camera have a way of letting you know the frame is ready, e.g. a callback? That would be better than waiting for a constant time, although that probably wouldn't describe the long waits. Is there a link to the camera docs? – Epicanthus

@Epicanthus As far as I can see, there is no such function. I can only copy the frame buffer or simply obtain a pointer to the framebuffer. Since the SDK that comes with the camera and includes the documentation is proprietary, I fear I don't have the rights to share it. – Kerman

The abnormal delay may be due to the GIL of python, as suggested in the documentation: "The Python global interpreter lock is released before calling any function exported by the libraries, and reacquired afterwards." – Jabiru

Did you solve your problem yet ? – Starve

@Starve No, I haven't been able to solve the problem. (But I also did not spend more time on it as this taking too much time anyways already) – Kerman