Display a numpy array in Kivy

Question

first of all, I'm totally new to kivy, so I'm struggling a bit.

I'm trying to display a numpy array in a kivy window. So far i figured out that this should work using the Texture Class (http://kivy.org/docs/api-kivy.graphics.texture.html).

As my numpy array changes from time to time, I'm trying to adjust the following code to my application.

# create a 64x64 texture, defaults to rgb / ubyte
texture = Texture.create(size=(64, 64))

# create 64x64 rgb tab, and fill with values from 0 to 255
# we'll have a gradient from black to white
size = 64 * 64 * 3
buf = [int(x * 255 / size) for x in range(size)]

# then, convert the array to a ubyte string
buf = b''.join(map(chr, buf))

# then blit the buffer
texture.blit_buffer(buf, colorfmt='rgb', bufferfmt='ubyte')

# that's all ! you can use it in your graphics now :)
# if self is a widget, you can do this
with self.canvas:
    Rectangle(texture=texture, pos=self.pos, size=(64, 64))

It seems that creating the texture and changing it works as it should, but i dont get, how to display the texture.

Can anybody explain to me, how to use the

with self.canvas:
    Rectangle(texture=texture, pos=self.pos, size=(64, 64))

in a way, that I get to see my picture/numpy array.

Thanks alot in advance! Holzroller

Edit: I figured out that using Kivy 1.8.0 and the Texture Class is a bit messy. So I upgraded to Kivy 1.9.0 via github (installing Kivy via apt-get in Ubuntu 14.04 LTS serves you the 1.8.0 version) and I get to see the Texture using the following code. I hope that helps people who are having the same problem as me.

from kivy.graphics.texture import Texture
from kivy.graphics import Rectangle
from kivy.uix.widget import Widget
from kivy.base import runTouchApp
from array import array
from kivy.core.window import Window


# create a 64x64 texture, defaults to rgb / ubyte
texture = Texture.create(size=(1280, 1024), colorfmt='rgb')

# create 64x64 rgb tab, and fill with values from 0 to 255
# we'll have a gradient from black to white
size = 1280 * 1024 * 3
buf = [int(x * 255 / size) for x in range(size)]

# then, convert the array to a ubyte string
arr = array('B', buf)
# buf = b''.join(map(chr, buf))

# then blit the buffer
texture.blit_buffer(arr, colorfmt='rgb', bufferfmt='ubyte')

# that's all ! you can use it in your graphics now :)
# if self is a widget, you can do this
root = Widget()
with root.canvas:
    Rectangle(texture=texture, pos=(0, 0), size=(1280*3, 1024*3))

runTouchApp(root)

Edit2: Basically I'm back to the original Problem: I have a numpy array (type 'numpy.ndarray'; dtype 'uint8') and I'm trying to convert it into a format, so that the texture will show me the image. I tried to break it down to the same way it is done in the example code i posted above. But i sadly doesn't work. I really do not know what I'm doing wrong here. (my numpy array is called im2 in the folling code)

list1 = numpy.array(im2).reshape(-1,).tolist()

arr = array('B', list1)

texture.blit_buffer(arr, colorfmt='rgb', bufferfmt='ubyte')

Downbow

Answer 1

Numpy have a tostring() attribute, that you could use directly, if the source array is uint8 type. You don't even need to reshape:

texture = Texture.create(size=(16, 16), colorfmt="rgb"))
arr = numpy.ndarray(shape=[16, 16, 3], dtype=numpy.uint8)
# fill your numpy array here
data = arr.tostring()
texture.blit_buffer(data, bufferfmt="ubyte", colorfmt="rgb"

About the issue you're talking in the comment, i see 2 points:

1. Ensure the callback from the ROS is called in the mainthread. Maybe the update is simply ignored.
2. When you manually change inplace the texture, the associated object that use it are not notified, you need to do it. Add a self.canvas.ask_update() to ensure the canvas redisplay at the next frame.

Counterpoise

anks, i tried that bevor. Maybe my fault is in updating that array. So I'll try to explain my whole application. I'm getting a Image via a ROS Node. I'm bringing that image back to a rgb format im2 = CvBridge().imgmsg_to_cv2(data, "rgb8") so I'm basically back to a numpy array. The ROS subscriber has a callback function where the texture is changed. But I dont get to see this change. Maybe it becomes more clear when you take a look at: (gist.github.com/Holzroller/1ec092ce798afd9c8b34) – Downbow

to 1. I don't think that the update is ignored, for that reason I added some print function to see if the callback is executed. Furthermore I was able to export the Image i got via ROS in a JPEG using OpenCV. to 2. I added self.canvas.ask_update() in every function, but sadly I dont get to see the image :(. Anyways, thank you so much for your support! gist.github.com/Holzroller/2f7f95b0958d6e406667 – Downbow

Not ignored in the python, ignored in GL as you are not in the opengl/main thread. But normally if you are not, it's crashing. I'm still don't understand how opencv/ros call the callback without beeing in the mainthread. – Counterpoise

Try to decorate the method with the @mainthread (from kivy.clock) or Clock.schedule_once the update when the callback is called from ROS (mainthread does that for you). – Counterpoise

Sorry, I cant really follow... what do you mean by "mainthread"? Should the ROS callback be in the MyApp class and from there call the Screen(widget) class? – Downbow

The main thread is the application itself, which Kivy control completely. You ask ROS for calling a python method, but how ROS is updated? They have an thread for sure, which call the python callback. So it can happen at any time (i make asumption about ROS, never used). Also you missed to call the super() in the widget init, and you call rospy.Subscriber everyframe, which seems not what you need. Usually, you subscribe once, and then you get update. – Counterpoise

There is too much to be discussed via comments here, prefer to use the Mailing list Kivy-users for guidance on your code, rather than SO. The question you asked got answered, but your issue on the code posted afterwards is not part of the initial question, so it's going outside the scope of SO :) – Counterpoise