ROS Image subscriber lag

Question

I am having some lag issues with a rospy subscriber listening to image messages.

Overview:

I have a rosbag streaming images to /camera/image_raw at 5Hz. I also have an image_view node for displaying the images for reference. This image_view shows them at 5Hz.

In my rospy subscriber (initialized with queue = 1), I also display the image (for comparing lag time against the image_view node). The subscriber subsequently does some heavy processing.

Expected result:

Since queue size is 1, the subscriber should process the latest frame, and skip all other frames in the meanwhile. Once it completes processing, it should then move on to the next latest frame. There should be no queuing of old frames. This would result in a choppy, but not laggy video (low fps, but no "delay" wrt rosbag stream, if that makes sense)

Actual result:

The subscriber is lagging behind the published stream. Specifically, the image_view node displays the images at 5Hz, and the subscriber seems to queue up all the images and processes them one by one, instead of just grabbing the latest image. The delay also grows over time. When I stop the rosbag stream, the subscriber continues to process images in the queue (even though queue = 1).

Note that if I change the subscriber to have a very large buffer size, as below, the expected behavior is produced:

self.subscriber = rospy.Subscriber("/camera/image_raw", Image, self.callback,  queue_size = 1, buff_size=2**24)

However, this is not a clean solution.

This issue has also been reported in the following links, where I found the buffer size solution. The official explanation hypothesizes that the publisher may actually be slowing down, but this is not the case since the image_view subscriber displays the images at 5Hz.

https://github.com/ros/ros_comm/issues/536, Ros subscriber not up to date, http://answers.ros.org/question/50112/unexpected-delay-in-rospy-subscriber/

Any help is appreciated. Thanks!

Code:

def callback(self, msg):
    print "Processing frame | Delay:%6.3f" % (rospy.Time.now() - msg.header.stamp).to_sec()
    orig_image = self.bridge.imgmsg_to_cv2(msg, "rgb8")
    if (self.is_image_show_on):
        bgr_image = cv2.cvtColor(orig_image, cv2.COLOR_RGB2BGR)
        cv2.imshow("Image window", bgr_image)
        cv2.waitKey(1)

    result = process(orig_image) #heavy processing task
    print result

Senior

did you find a solution? I've noticed a significant delay when using multiple subscriber for the same image topic. – Conard

Answer 1

first of all if You Intend to Calculate Delay In Your topic Stream You Are going The Wrong Way since Both image_view node and cv_bridge+opencv use Lots of Resources To Show The images which cause some lags themselves.

After That Subscribing To An Image Topic From 2 Different Node is still unstable(In My Distro which is indigo) unless you change transport hint on publisher side

What i suggest to do is stop on of the subscriber nodes and check that the image is streaming correctly(in Your Code especially) and then use rostopic delay someTopic to show you the delay Your are getting. If the Problem Still presists You may change the transport_hint in publisher to UDP (not sure possible to do it in rosbag but in real driver quite possible)

Managerial

Answer 2

When working with large size messages like images or point clouds, the right way to go is to use Nodelets.

Feeney

Answer 3

You should not do "heavy processing tasks" inside the callback. This just blocks the ros executor unnesessarily. You might wanna read this and switch to multithreaded executors: http://wiki.ros.org/roscpp/Overview/Callbacks%20and%20Spinning

To avoid long running callbacks, rather store the image into a member and do the heavy processing in a different part of the program.

Something like:

def callback(self, msg):
    self.img = msg

def process(self):
    print "Processing frame | Delay:%6.3f" % (rospy.Time.now() - msg.header.stamp).to_sec()
    orig_image = self.bridge.imgmsg_to_cv2(self.img, "rgb8")
    if (self.is_image_show_on):
        bgr_image = cv2.cvtColor(orig_image, cv2.COLOR_RGB2BGR)
        cv2.imshow("Image window", bgr_image)
        cv2.waitKey(1)
    
    result = process(orig_image)

(code is untested and written out of my mind)

Knitter