Display OpenCV Mat with JavaFX

Question

I would like to display Mat objects from OpenCV directly with JavaFX. I have seen that it is possible to convert a Mat object into a BufferedImage. But as far as I know you can't display a BufferedImage with JavaFX, so another conversion would have to take place.

Is there a way to directly convert it into a data structure that is displayable by JavaFX?

Roseleeroselia

It's probably possible to convert directly to FX, using a WritableImage. Either by copying pixels from the Mat to the WritableImage's PixelWriter, or by implementing a PixelReader backed by the Mat, and use that for instantiating the WritableImage directly. – Reis

Answer 1

I have found a direct way to convert a Mat object to a JavaFX Image object.

MatOfByte byteMat = new MatOfByte();
Highgui.imencode(".bmp", mat, byteMat);
return new Image(new ByteArrayInputStream(byteMat.toArray()));

You can also encode it to .jpg but .bmp is faster.

Roseleeroselia

Why my OpenCV doesn't come with HighGUI library/class ? – Grau

Answer 2

TomTom's answer was very helpful and solved the problem, but Highgui does no longer have any bindings in java.

As of OpenCV 3.0.0, the up-to-date code is more like:

MatOfByte byteMat = new MatOfByte();
Imgcodecs.imencode(".bmp", mat, byteMat);
return new Image(new ByteArrayInputStream(byteMat.toArray()));

Nummulite

Your answer is the solution for OpenCV version 3. Earlier versions use Highgui. – Eyeopener

Thank you for your precision, I just edited and corrected my answer. – Nummulite

Answer 3

The silly way to do this is to convert the Mat to a BufferedImage, and then that to an Image so that it can be displayed inside an ImageView:

Mat >> BufferedImage >> Image >> ImageView

Assuming you know how to do the 1st conversion, the rest would be something like this:

import javafx.embed.swing.SwingFXUtils;
import javafx.scene.image.Image;
import javafx.scene.image.ImageView;

Image image = SwingFXUtils.toFXImage(bufImage, null);

ImageView imgView = new ImageView();
imgView.setImage(image);

I haven't tested the code, but that's the general idea.

Orpheus

Answer 4

Instead of encoding and decoding to get the byte array you need, you can just declare an array and use the get method. This is Scala code, but hopefully it's intelligible:

val arr = new Array[Byte](w * h * 3)
mat.get(0, 0, arr)
pw.setPixels(0, 0, w, h, PixelFormat.getByteRgbInstance, arr, 0, w * 3)

This code declares the array arr of type byte[] with the size corresponding to an image of h by w and 3 channels. Then the data is copied from the mat object into the array, and passed to the setPixels method from the PixelWriter object pw.

Engine