How can I access the Kinect using Java?

Question

I am currently in a Computer Vision course and for my final project I am going to make a small game that interacts with the Kinect.

Now I want to make this game in Java as I have never really ventured into making a game before and I am very comfortable with the Language. But I cannot seem to find a way to just access the Depth data (I just need to pack it into some Java Data structure) and the RGB data (same idea).

Right now, I have a way to 'talk' to the Kinect. I followed this tutorial and can detect the kinect using the command glview in terminal.

I have a thing called Processing which promised to be a Java library to let me use the Kinect but it is using it's own language which is 'built' using Java so it's not really Java which means I can't use all the GUI and normal Java features.

What I want to do is somehow call some import random.Kinect.code; at the top of the Java program that lets me use the most basic Kinect features, like getDepth() or getRGB() and do everything else in Java.

Does anyone know a way I can do this? (BTW I'm running MAC OSX Lion)

Venipuncture

Is this like interfacing using java? – Controvert

If OpenKinect is your thing, JavaCVpro should work... – Refuel

Answer 1

If you're using Processing I recommend using one of the available wrappers:

• Daniel Shiffman's OpenKinect Processing (using openkinect/libfreenect)
• SimpleOpenNI - Processing OpenNI wrapper

Both are well documented and include plenty of helpful examples.

Jany

well that's the thing, I have Processing but it is some weird form of Java, do you know whether it permits using the full spectrum of the Java language on top of it's weird different syntax/shortcuts? – Venipuncture

I've used these libraries in the Processing IDE and eclipse as well on osx 10.6. What's your setup ? You should be fine to either use Processing on it's own or use it as a library in a Java project. For eclipse, have a look at Proclipsing or simply create a Java project and use Processing's core.jar if you want to deal with it 'manually'. – Jany

I just want to program it with a simple text editor and the command line, I made some headway by discovering how to tap into the openkinect (libfreenect) java wrapper. Which basically consisted of compiling somehow including two of their jars. <- As you can see I'm fairly new to most of these programming concepts.. Would I be able to somehow tap into Processing's 'core.jar' like you said in a similar manner to the one I just described and just use a couple of their methods in my java program? If so could you try to help me out and explain how I might do that? Thanks for the help so far! – Venipuncture

Try this very basic setup: 1. Download Processing and run once (so you have a Processing folder in your Documents folder). 2. Download the Processing OpenKinect wrapper and unzip it in Documents/Processing/libraries. 3. Run Processing and go to File > Examples > Contributed Libraries > openkinect > PointCloud or open the PointCloud.pde file from Documents/Processing/libraries/openkinect/examples/PointCloud. This should take minutes to test. The Processing IDE is a simple text editor... – Jany

You won't need the command line, you shouldn't need to compile any .jars - the aim of Processing is to things easy to use. Still, if you want to do things from scratch (do you need to though ? ), regarding core.jar, you probably want to create a class that extends processing.core.PApplet and override the setup() and draw() methods to get started. PApplet itself extends the java Applet class if that helps. – Jany

Answer 2

I had exactly the same problem. I wanted to use Kinect in Java for my Computer Vision research, and I had a Mac OSX Lion like you. The solution that I found is to run a virtual copy of Windows 7 in my Mac (using VMWare or Parallels or Microsoft HyperV) and then use the open source J4K Java for Kinect library, which is a Java binding for the Kinect SDK using the Java Native Interface (JNI). Of course you can also use this open source library directly in computers that run Windows.

The J4K Java library for Kinect also has a tutorial on how to write a kinect based application in just 10 lines of Java code.

There are also several source code examples that shows you how to read depth, video, and skeleton data from the Kinect and use them in Java classes. You can also use multiple Kinects, animate avatars, and other examples. The link to the source code examples is here .

A screenshot of the open source Kinect Viewer, which is useful for Computer Vision applications like yours is below:

I hope that this information helped.

Silkstocking

Does this not slow down the image processing? I assume deferring the incoming data through a virtual machine is gonna cost performance, right? – Perfectly

I have used J4K library in Java 7 with Intel i5 and very intensive image processing was achieved in real time (30fps). @Perfectly you can take a look at actual examples Demo1, Demo2, and Demo3. – Silkstocking

Answer 3

I will give you some ideas, but I admit - I did no try them with Kinect.
You question can be categorized under questions of "How to access some hardware via Java". The tutorial you presented us presents libraries for accessing Kinect, and even wrappers for Python.
What you can do is see how the Python code implemented the wrappers, and use either JNI/JNA to build your own wrappers.
In addition, I looked into the JNect project, mentioned here in another answer, posted by @npinti.
The eclipse plugin has two parts of it -
the JNI code, which wraps theo riginal SDK code, and the Ecipse RCP code for the eclipse plugin itself.
I would recommend you to ignore the Eclipse RCP part, and focus on their JNI implementation as a reference to what you need to develop your own JNI wrapper.
Even if this is windows-based implementation (i.e - Kinect SDK works on windows, and you need to do run your code on mac, for example) - I think that by reading their code + some JNI tutorials + the python wrappers at the link you provided you will understand how to create a JNI wrapper to the library presented in the link you provided.
In addition, consider using JNA and not JNI as a way to communicate with hardware, I find it easier.
One last option is maybe to use the Java Script Engine and to invoke Python code (the python wrappers) from your java vode - not sure how much it will help you, but you can give it a shot.

Kaph

Unfortunately it seems I missed the whole Windows 7 Only deal... To deal with this the OP might consider using a Windows Based Virtual Machine. – Parsifal