Kinect for Windows gesture recognition

Question

I have been looking at the Kinect for Windows release notes and features, since I want to incorporate gesture recognition in my project as well.

At the above page, the first line mentions that "The Kinect for Windows SDK enables developers to create applications that support gesture and voice recognition, ". Voice recognition API is available with the SDK and readily can be used. However, I don't think there are any gesture recognition APIs available in the SDK. The APIs of Skeleton Tracking are there to be used readily but then they have to be tailored with to get gesture recognition.

I have seen videos of Windows Media Center beng controlled by gestures etc. and other applications too. I wonder if all these applications are custom built and have to write their own gesture recognition code.

Currently, in my project I am using Kinect DTW Gesture Recognition from Codeplex. I am having two issues with this -> 1) Looks very performance hogging, and on enabling this with my app, my app throws OutofMemory exception after some time (PC specs are pretty high). 2) Can't say much about the robustness of the system. Works at times for some people and not for others.

I thought if the APIs would have been inbuilt, it would have been good to switch to these. Are these available or else what's the resolution?

Dvina

Profile the cause of the exception, then report back, your question as written is vague and would require us to find the problem for you. – Howrah

@Ramhound: My question is mainly asking if there's an inbuilt gesture recognition system in the Kinect SDK as well? – Dvina

looking at that codeplex project, it looks like an idea that fizzled out. have you inspected the SDK's namespaces and objects? you may have to create some kind of Gesture class which defines the position of each limb and whatnot.. – Chiseler

maybe this Question will help – Chiseler

Answer 1

I'm actually doing this right now for a school project. We had to create our own Gesture Recognition module. There's not anything in the API that will do it for you, but it provides you with all the pieces you'll need to build the capability up.

This article was a big help, http://blogs.msdn.com/b/mcsuksoldev/archive/2011/08/08/writing-a-gesture-service-with-the-kinect-for-windows-sdk.aspx. It talks about how to break down gestures. If you only have a handful of gestures that you can hard-code, its trivial. We needed the ability to load in and recognize user defined gestures, but this article describes the basic framework we used to get there.

Kablesh

Answer 2

Have a look here http://www.exceptontuesdays.com/gestures-with-microsoft-kinect-for-windows-sdk-v1-5/

Source code is available on that page.

It recognizes the following

 switch (e.GestureType)
            {
                case GestureType.Menu:
                    Debug.WriteLine("Menu");
                    Gesture = "Menu";
                    break;
                case GestureType.WaveRight:
                    Debug.WriteLine("Wave Right");
                    Gesture = "Wave Right";
                    break;
                case GestureType.WaveLeft:
                    Debug.WriteLine("Wave Left");
                    Gesture = "Wave Left";
                    break;
                case GestureType.JoinedHands:
                    Debug.WriteLine("Joined Hands");
                    Gesture = "Joined Hands";
                    break;
                case GestureType.SwipeLeft:
                    Debug.WriteLine("Swipe Left");
                    Gesture = "Swipe Left";
                    break;
                case GestureType.SwipeRight:
                    Debug.WriteLine("Swipe Right");
                    Gesture = "Swipe Right";
                    break;
                case GestureType.ZoomIn:
                    Debug.WriteLine("Zoom In");
                    Gesture = "Zoom In";
                    break;
                case GestureType.ZoomOut:
                    Debug.WriteLine("Zoom Out");
                    Gesture = "Zoom Out";
                    break;

                default:
                    break;

Biology

@dimi Anyone with the updated link? I would like to implment zoom in and zoom out feature – Eldwon

It seems to be that this github repo is from the link above. Check out github.com/EvilClosetMonkey – Thyrotoxicosis

Answer 3

I am also developing an application with gesture recognition as a assignment in University. After searching for good books I stumbled across these two, both written by Microsoft researchers:

(1) http://www.amazon.de/Programming-Kinect-Windows-Software-Development/dp/0735666814
(2) http://www.amazon.de/Kinect-Windows-SDK-Programming-Guide/dp/1849692386/ref=sr_1_10?s=books-intl-de&ie=UTF8&qid=1393944100&sr=1-10&keywords=kinect

They describe general techniques to make gesture recognition possible with the Kinect.

Hereupon

Answer 4

Try this FAAST API. It's an API where it can recognize gestures of body and gives a respective input interrupt. Here you need to assign a particular key for a particular action. For example: Suppose I use 'W' key for forward movement then I can assign this key for a particular gesture using this API.

Satirical