How can you implement non-standard Leap Motion gestures?

Question

The Leap Motion API only supports four standard gestures: circle, swipe, key tap and screen tap. In my application I need other gestures, but I do not know how can I add them or if it is even possible to add more gestures. I read the API and it was no help.

In my application I want to allow the user to hold an object and drag it. Is that possible using the Leap Motion API? If so, how can I do this?

Pouter

First: What's your langage you're using to develop? You have to study exactly the x/y/z axsis of your gesture and analyze them. – Fiveandten

As Larme says, you're going to need to provide a little more information here for us to help you. Which Leap Motion API are you using (JS, Objective-C, etc.)? How do you want people to hold this object (finger, grasping hand)? Also, you should read that text you're presented with, rather than trying to game the system by pasting into your question. – Armillas

OK, I reworded your question and removed the garbage at the end. It's still a little broad, but it's closer to being answerable. It still would help to have a few more details, though. – Armillas

The gestures in the Leap Motion API aren't extensible. You will have to create your own gesture recognition system from start to finish to recognize new ones. – Quillan

Answer 1

You will need to create your own methods to recognize the starting point, the process, and the ending point of a gesture.

Starting point: How will your program recognize you are trying to hold something? A simple gesture I can think of is 2 fingers linked with 1 palm. So in the frame, if you see 2 fingers linked with 1 palm and the fingers are maybe 10-20 mm apart, you can recognize it as a gesture to hold something. When these conditions are fulfilled, the program will recognize the gesture and you can write some code inside these conditions.

For a very ugly example in C#:

Starting point: Boolean gesture_detected = false;

Frame frame = controller.Frame();
HandList hands = controller.Hands;

if (hands.Count == 1)
{
   foreach (Hand hand in hands)
   {
       if (hand.fingers.Count == 2)
       {
           int fingerA_x,fingerB_x;
           foreach (Finger finger in hand.fingers)
           {
               if(fingerA_x == 0)
               {
                   fingerA_x = finger.x;
               } else
               {
                   fingerB_x = finger.x;
               }
           }
       }
   }

   if((fingerA_x - fingerB_x) < 20)
   {
         //Gesture is detected. Do something...
         gesture_detected = true;
   }
}

Process: What is your gesture trying to do? If you want to move around, you will have to call a mouse method to do a drag. Search for the method mouse_event() in C++ under PInvoke using the event MOUSEEVENTF_LEFTDOWN.

Ending point: After you finish dragging, you need to call a mouse method event like MOUSEEVENTF_LEFTUP to simulate a mouse drag that has finished. But how will your program detect when you should stop the drag? Most logical way is if the gesture is no longer being detected in the frame. So write an else condition to process the alternate scenario.

       if (!gesture_detected)
       {
            // Do something
       }

Gib

Answer 2

Did wonder myself at a point in time if there was a need to derive some Leap Motion classe(s) in order to define custom classes as with other API's, turns out you do not have to. Here below is a short example in C++, the Leap Motion native language, of a custom gesture definition, the Lift gesture which lifts in the air all kinematic objects in your world.

The gesture definition calls for 2 visible hands, both open flat with palms up and moving up slower than the pre-defined (currently deprecated) Leap Motion Swipe gesture, so there is no confusion between the 2 gestures in case the Swipe gesture has been enabled during start-up in the Leap Motion callback function void GestureListener::onConnect(const Leap::Controller& controller).

As seen from the example, the gesture definition imposes constraints on both hands normals and velocities, so it won't be detected at random but not too many constraints, so it can still be executed with some reasonable effort.

// lift gesture: 2 hands open flat, both palms up and moving up slowly
void LeapMotion::liftGesture(int numberHands, std::vector<glm::vec3> palmNormals, std::vector<glm::vec3> palmVelocities) {

    if ((numberHands == 2) &&
        (palmNormals[0].x < 0.4f) && (palmNormals[1].x < 0.4f) &&
        (palmNormals[0].y > 0.9f) && (palmNormals[1].y > 0.9f) &&
        (palmNormals[0].z < 0.4f) && (palmNormals[1].z < 0.4f) &&
        (palmVelocities[0].z > 50.0f) && (palmVelocities[1].z > 50.0f) &&
        (palmVelocities[0].z < 300.0f) && (palmVelocities[1].z < 300.0f)) {

        m_gesture = LIFT;
        logFileStderr(VERBOSE, "\nLift gesture...\n");
    }
}

Cioffred