How could it work to use multiple cursors on one Windows client?

Question

I recently searched the web for multi-cursor solutions and found a bunch of possibilities to emulate cursors in a given window, like MultiPoint SDK, but no solutions where one can use multiple cursors on the whole desktop (and thus not restricted to one application).

I almost gave up, thinking the Windows architecture makes it impossible, but then I found MultiMouse, stating in this video on YouTube, it would at least make it possible to use different cursors for different users over the whole desktop. If that is true, it follows that one could emulate multiple users and thereby making a multiple cursors solution possible.

So my questions are:

• Does someone here use Multimouse and is it really working as described?
• How does it work, can someone use the windows api, or needs to reconfigure Windows to do so?
• Have you any ideas on how to draw and command multiple cursors on Windows 7? Problem is, there is only one mouse pointer. Can someone change that, or what are remote applications doing? I mean, it would suffice to just show the cursors (globally) and control them automatically.
• I'm free to use any interop techniques, no matter how dirty they are, since it suffices if my app runs only on Windows 7. Does someone know windows libraries which could provide such functionality? Sadly, the windows api is poorly documented.

Edit: In the following I will explain how windows messages partially work, but why they don't do in general. First, we take a flash client as an example, that is a handle with a child, and it's child is a flash object. Whenever a click is performed anywhere at the flash area, the following windows messages show up:

<000001> 00100354 S WM_PARENTNOTIFY fwEvent:WM_LBUTTON xPos:[x] yPos:[y]

<000002> 00100354 R WM_PARENTNOTIFY

<000003> 001B09D6 S WM_MOUSEACTIVATE hwndTopLevel:00100354 nHittest:HTCLIENT uMsg:WM_LBUTTONDOWN

<000004> 00100354 S WM_MOUSEACTIVATE hwndTopLevel:00100354 nHittest:HTCLIENT uMsg:WM_LBUTTONDOWN

<000005> 00100354 R WM_MOUSEACTIVATE fuActivate:MA_ACTIVATE

<000006> 001B09D6 R WM_MOUSEACTIVATE fuActivate:MA_ACTIVATE

<000007> 001B09D6 P WM_LBUTTONDOWN fwKeys:MK_LBUTTON xPos:[x] yPos:[y]

<000008> 001B09D6 P WM_LBUTTONUP fwKeys:0000 xPos:[x] yPos:[y]

In this example 00100354 is the parent and 001B09D6 is the child (flash object). So it happens, that the parent tells the child to click at a certain position ([x],[y]). But a faking click solution working with windows messages could never know which handle it has to click. If the clicks or notifications are only sent to 00100354, nothing happens. What we are trying to accomplish are the messages <000007> and <000008>, but we can't, except by spamming messages to all descendants of the foreground handle, including the foreground handle itself. But this could also lead to clicks we don't want. We cannot perform a hit test since the target application may have e.g. transparent handles in front of the target handle. Note that one also needs to calculate new relative coordinates if a child handle is not positioned at (0,0) of its parent.

To realize communication as shown above, one can use Win32 calls, like this:

[DllImport("User32.dll", SetLastError = true)]
public static extern IntPtr SendMessage(IntPtr hWnd, WM Msg, IntPtr wParam, IntPtr lParam);

[return: MarshalAs(UnmanagedType.Bool)]
[DllImport("user32.dll", SetLastError = true)]
public static extern bool PostMessage(IntPtr hWnd, WM Msg, IntPtr wParam, IntPtr lParam);

[...]

public static IntPtr ToWMCoords(Point pt)
{
    return (IntPtr)(((uint)pt.Y << 16) | (ushort)pt.X);
}

[...]

 foreach (Point pt in coordinates)
 {
      Console.Write("Message-Clicking at " + pt + "\n");
      IntPtr mousePos = ToWMCoords(pt);

      //# Get flash handle
      //## Variation, assuming the client has always only 1 child with class name "MacromediaFlashPlayerActiveX"
      IntPtr flashHwnd = Interop.GetChildWindows(hwnd).Where(x => Interop.GetClassNameOfHandle(x) == "MacromediaFlashPlayerActiveX").Single();
                    
      //# Simulating a click
      //*1: Messages getting to handle of class "#32770 (Dialog)"
      //*2: Messages getting to handle of class  "MacromediaFlashPlayerActiveX"

      Interop.User32.SendMessage(hwnd, Interop.WM.PARENTNOTIFY, (IntPtr)0x201, mousePos);//*1
      Interop.User32.SendMessage(flashHwnd, Interop.WM.MOUSEACTIVATE, hWnd, (IntPtr)0x2010001);//*2
      Interop.User32.SendMessage(hwnd, Interop.WM.MOUSEACTIVATE, hWnd, (IntPtr)0x2010001);//*1
      bool lBtnDown = Interop.User32.PostMessage(flashHwnd, Interop.WM.LBUTTONDOWN, (IntPtr)Interop.MK.LBUTTON, mousePos);//*2
      bool lBtnUp = Interop.User32.PostMessage(flashHwnd, Interop.WM.LBUTTONUP, IntPtr.Zero, mousePos);//*2
      Console.Write("WM_LBUTTONDOWN " + (lBtnDown ? "success" : "fail") + ", WM_LBUTTONUP " + (lBtnDown ? "success" : "fail") + "\n");
  }

In summary, faking mouse input using windows messages appears to be improperly or impossible, depending on the target application.

Mickel

Upon reading the manual of Multimouse - please note that I did not use the actual product - I came to the strong suspicion that it only fakes the additional cursors using one of the techniques I have described in my answer. But once again it is just a suspicion. – Drus

I also started to think that :( But it would make their tool quite crappy. I think a good way to check that would be, to use a drawing tool as mspaint and to draw in there by more than one cursor at a time.. the tool is not faking if it works (even if mspaint is acting weird due to so much input information). If it works only with a single user drawing, probably the real cursor would be used whenever a single user is performing an action. If it never works, its all a big fake, using simply windows messages. – Mickel

I think the idea in general is not bad, but I would use some other tool since I'm not sure if MsPaint uses the multipoint SDK controls which could handle multiple mouse inputs, however have no use for you since their scope is the application they are built into (and they are quite crappy even there: i.e. embedded controls like scrollbars of text boxes does not work). – Drus

Answer 1

If you are looking for a solution to have a pointer-like object that is moved around on the desktop, I might have an idea for a "dirty" solution, but I wish to warn you that this should be your last choice. It is real dirty and ill-advised. (I did something like this when I was a real beginner and I would never do it again.)

---

So here is the deal:

You might create a topmost little form which is mouse pointer-shaped. It then can be programmatically moved around the screen. The clicks/doubleclicks/hovering can be emulated through accessibility or through windows messages (both solutions have their own caveeats, so choose wisely).

Accessibility provides a somewhat standardized way to access forms and controls, and would be the recommended solution but there is a problem: Not all windows programs support accessibility (especially older software and games), so some of them will not respond to the fake pointer.

Windows messages are the unmanaged Win32 way to go if you want something like that. In that case you can send mouse messages (WM_CAPTURECHANGED WM_LBUTTONDBLCLK WM_LBUTTONDOWN WM_LBUTTONUP WM_MBUTTONDOWN WM_MBUTTONUP WM_MOUSEMOVE WM_MOUSEWHEEL WM_RBUTTONDOWN WM_RBUTTONUP) to the form under your fake cursor. This solution has the caveeat, that you have to invoke the non-managed postmessage API.

---

Whatever solution form the above listed you are using be prepared that some programs manage the cursor directly, so some software will act weird.

Drus

But really... only do this if there is no other solution. I don't know if windows supports multiple cursors and my solution only helps faking one. – Drus

Thanks for the answer, but that is not what I looked for. It is rather the trivial fake solution which I'm trying to avoid by asking this question. There are even more problems with that solution, for example Windows messages cannot simulate Windows input, as explained here. – Mickel

@Mickel - That is why I advised to use any other solution if there is any. (As far as I know there is not.) Also the Old New Thing article you have linked only mentions character input and it is for a reason. Most applications handle mouse events through windows messages therefore can work with my solution. It is a pretty common practice to handle keyboard input directly, but not so common to directly handle mouse. – Drus

@Mickel - Also if anyone comes up with a better solution could you notice me? I just became curious. (Thanks in advance.) – Drus

It is analogous for mouse input, as edited into my question. Sure, if I find any better solution, I'll post my own answer here. By the way: If one clicks with a cursor, a message is posted, not send. That means, to fake a mouse click, you need to use PostMessage, not SendMessage (at least on my Windows 7 x64 system). – Mickel

@Mickel - I never use the SendMessage() API since it blocks the sender thread. I believe(/hope) nobody does. I wrote "send" as a general term not as a reference to the mentioned API. I used general terms instead of demo code because I intended my answer to people with general knowledge of Win32 programing. BTW when I think about it it might have a point since you are emulating a mouse which is a sequential input device. But I'm not sure about that last thing it is just a n idea formed at this very moment. :D – Drus