Generate and post Multitouch-Events in OS X to control the mac using an external camera
Asked Answered
H

1

7

I am currently working on a research project for my university. The goal is to control a Mac using the Microsoft Kinect camera. Another student is writing the Kinect driver (which will be mounted somewhere on the ceiling or the wall behind the Mac and which outputs the position of all fingers on the Macs screen).

It is my responsibility to use that finger-positions and react on them. The goal is to use one single finger to control the mouse and react on multiple fingers the very same way, like they are on the trackpad.

I thought that this is going to be easy and straight forward, but its not. It is actually very easy to control the mouse cursor using one finger (using CGEvent), but unfortunately there is no public API for creating and posting Multitouch-Gestures to the system.

I've done a lot of research, including catching all CGEvents using an event tap at the lowest possible position and trying to disassemble them, but no real progress so far.

Than I stumbled over this and realized, that even the lowest position for an event tap is not deep enough: Extending Functionality of Magic Mouse: Do I Need a kext?

When I got it right, the built-in Trackpad (and the MagicMouse and the MagicTrackpad) communicates over a KEXT-Kernel-Extension with the private MultitouchSupport-framework, which is generating and posting the incoming data in some way to the OS.

So I would need to use private APIs from the MultitouchSupport.framework to do the very same thing like the Trackpad does, right? Or would I need to write a KEXT-Extension?

And if I need to use the MultitouchSupport-framework: How can I disassemble it to get the private APIs? (I know class-dump, but that only works on Objective-C-frameworks, which this framework is not)

Many thanks for any response! NexD.

Helenahelene answered 14/7, 2011 at 19:54 Comment(2)
You could try something like the suggestion here https://mcmap.net/q/1555499/-is-there-a-way-to-trigger-gesture-events-on-mac-os-x Did you come up with another solution?Unhurried
Has anyone found a solution for this??? I am struggling with a similar problem a couple of days already, but nothing seems to work :(Farfamed
D
0

"The goal is to use one single finger to control the mouse and react on multiple fingers the very same way" here if I understand what you are trying to do is you try to track fingers from Kinect. But the thing is Kinect captures only major body joints. But you can do this with other third party libraries I guess. Here is a sample project I saw. But its for windows. Just try to get the big picture there http://channel9.msdn.com/coding4fun/kinect/Finger-Tracking-with-Kinect-SDK-and-the-Kinect-for-XBox-360-Device

Duvalier answered 9/11, 2013 at 15:48 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.