Turtlebot subscriber pointcloud2 shows color in Gazebo simulator but not in robot

Question

I am subscribing to topic "/camera/depth/points" and message PointCloud2 on a turtlebot (deep learning version) with ASUS Xtion PRO LIVE camera.

I have used the python script below under the gazebo simulator environment and i can receive x, y, z and rgb values successfully.

However, when i run it in the robot, the rgb values are missing.

Is this a problem of my turtlebot version, or camera or is it that i have to specify somewhere that i want to receive PointCloud2 type="XYZRGB"? or is it a sync problem? Any clues please thanks!

#!/usr/bin/env python
import rospy
import struct
import ctypes
import sensor_msgs.point_cloud2 as pc2
from sensor_msgs.msg import PointCloud2

file  = open('workfile.txt', 'w')

def callback(msg):

    data_out = pc2.read_points(msg, skip_nans=True)

    loop = True
    while loop:
        try:
            int_data = next(data_out)
            s = struct.pack('>f' ,int_data[3])
            i = struct.unpack('>l',s)[0]
            pack = ctypes.c_uint32(i).value

            r = (pack & 0x00FF0000)>> 16
            g = (pack & 0x0000FF00)>> 8
            b = (pack & 0x000000FF)

            file.write(str(int_data[0])+","+str(int_data[1])+","+str(int_data[2])+","+str(r)+","+str(g)+","+str(b)+"\n")

        except Exception as e:
            rospy.loginfo(e.message)
            loop = False
            file.flush
            file.close

def listener():

    rospy.init_node('writeCloudsToFile', anonymous=True)
    rospy.Subscriber("/camera/depth/points", PointCloud2, callback)
    rospy.spin()

if __name__ == '__main__':
    listener()

Burdensome

ASUS Xtion PRO LIVE – Burdensome

I think i have tried depth-registered as well but i cant remember now, i will check this once – Burdensome

This might help, i have tried with openni but couldn't get it to work not even using depth-registered. But i dont think i set the parameter as indicated in your link depth_registration:=true so i will try this tomorrow morning. 1 question, is using openni for this the most normal approach? – Burdensome

@peter Brittain well, you nailed it. /camera/depth/points only shows xyz, whereas /camera/depth_registered/points shows the combination of xyz with rgb. Funny thing is why is it different in gazebo.. well, never mind. If you post it as an answer i will accept it. – Burdensome

Thanks! I've now tidied up my comments and moved all the content into the answer. – Cheree

Answer 1

The contents of Published topics are determined by the software that provides them - i.e. the drivers for your camera. To fix this you therefore need to get the right driver and use the topic that it says contains the required information.

You can find recommended drivers for your cameras on the ROS wiki or on some community websites - like this. In your case, the ASUS devices should use openni2 and set depth_registration:=true - as documented here.

At this point, /camera/depth_registered/points should now show the combined xyz and RGB point cloud. To use it, your new listener() code should look something like this:

def listener():
    rospy.init_node('writeCloudsToFile', anonymous=True)
    # Note the change to the topic name
    rospy.Subscriber("/camera/depth_registered/points", PointCloud2, callback)
    rospy.spin()

Cheree