Simulate mouse click in MSPaint

Question

I have a console application which should paint a random picture in MSPaint (mouse down -> let the cursor randomly paint something -> mouse up. This is what I have so far (I added comments to the Main method for better understanding what I want to achieve):

[DllImport("user32.dll", CallingConvention = CallingConvention.StdCall)]
public static extern void mouse_event(long dwFlags, uint dx, uint dy, long cButtons, long dwExtraInfo);
private const int MOUSEEVENTF_LEFTDOWN = 0x201;
private const int MOUSEEVENTF_LEFTUP = 0x202;
private const uint MK_LBUTTON = 0x0001;

public delegate bool EnumWindowsProc(IntPtr hWnd, IntPtr parameter);

[DllImport("user32.dll", SetLastError = true)]
static extern IntPtr FindWindow(string lpClassName, string lpWindowName);

[DllImport("user32.dll", SetLastError = true)]
public static extern IntPtr FindWindowEx(IntPtr parentHandle, IntPtr childAfter, string className, string windowTitle);

[DllImport("user32.dll", CharSet = CharSet.Auto)]
static extern IntPtr SendMessage(IntPtr hWnd, UInt32 Msg, IntPtr wParam, IntPtr lParam);

[DllImport("user32.dll", SetLastError = true)]
public static extern bool EnumChildWindows(IntPtr hwndParent, EnumWindowsProc lpEnumFunc, IntPtr lParam);

static IntPtr childWindow;

private static bool EnumWindow(IntPtr handle, IntPtr pointer)
{
    childWindow = handle;
    return false;
}

public static void Main(string[] args)
{
    OpenPaint(); // Method that opens MSPaint
    IntPtr hwndMain = FindWindow("mspaint", null);
    IntPtr hwndView = FindWindowEx(hwndMain, IntPtr.Zero, "MSPaintView", null);
    // Getting the child windows of MSPaintView because it seems that the class name of the child isn't constant
    EnumChildWindows(hwndView, new EnumWindowsProc(EnumWindow), IntPtr.Zero);
    Random random = new Random();
    Thread.Sleep(500);

    // Simulate a left click without releasing it
    SendMessage(childWindow, MOUSEEVENTF_LEFTDOWN, new IntPtr(MK_LBUTTON), CreateLParam(random.Next(10, 930), random.Next(150, 880)));
    for (int counter = 0; counter < 50; counter++)
    {
        // Change the cursor position to a random point in the paint area
        Cursor.Position = new Point(random.Next(10, 930), random.Next(150, 880));
        Thread.Sleep(100);
    }
    // Release the left click
    SendMessage(childWindow, MOUSEEVENTF_LEFTUP, new IntPtr(MK_LBUTTON), CreateLParam(random.Next(10, 930), random.Next(150, 880)));
}

I got this code for the click simulation from here.

The click gets simulated but it doesn't paint anything. It seems that the click doesn't work inside MSPaint. The cursor changes to the "cross" of MSPaint but as I mentioned...the click doesn't seem to work.

FindWindow sets the value of hwndMain to value 0. Changing the parameter mspaint to MSPaintApp doesn't change anything. The value of hwndMain stays 0.

If it helps, here is my OpenPaint() method:

private static void OpenPaint()
{
    Process.process = new Process();
    process.StartInfo.FileName = "mspaint.exe";
    process.StartInfo.WindowStyle = "ProcessWindowStyle.Maximized;
    process.Start();
}

What am I doing wrong?

Cateran

Step one: Try if it works better with some other application than Paint, then report! – Rhinoplasty

Hi! I like this question and am curious about it - did you already find a answer to it or is this still open? If you haven't found a answer now, I'll try it this evening myself. – Swords

@Swords I didn't find an answer yet. I can't check because I'm ill atm.. – Cateran

Hi @diiN_ just posted an answer ;) – Swords

Answer 1

As is promised I tested it myself yesterday - to be honest my cursor just moved, but not in the window, and without any affect - as I did debug I saw that the var hwndMain = FindWindow("mspaint ", null); was value 0. I though that this has to be the problem so I did take a look at the other stackoverflow topic, you got your code from. I recognized the solution was using a different window-name they were looking after at FindWindow() - so I did try.

var hwndMain = FindWindow("MSPaintApp", null);

After changing the methodcall it worked out for me - though - after moving the MsPaint there cursor was still in the original open Position - you might want to think about that and ask the window for it's position maybe. Did the Name maybe changed with Win7 / 8 / 10 ?

Edit:

On Windows 10 the name for paint seems to be changed - so I guess you still have problems on getting the right window handle - this was proved wrong by Hans Passant, who explained nicely whats the Problem with the handler (link below). One way to solve this would be to get you handler from the process itself instead of getting it from FindWindow()

I suggest you change your OpenPaint() like this:

 private IntPtr OpenPaint()
 {
    Process process = new Process();
    process.StartInfo.FileName = "mspaint.exe";
    process.StartInfo.WindowStyle = ProcessWindowStyle.Maximized;
    process.Start();
    // As suggested by Thread Owner Thread.Sleep so we get no probs with the handle not set yet
    //Thread.Sleep(500); - bad as suggested by @Hans Passant in his post below, 
    // a much better approach would be WaitForInputIdle() as he describes it in his post.         
    process.WaitForInputIdle();
    return process.MainWindowHandle;
 }

Link to Hans Passant decription for the explaination why Thread.Sleep() just is a bad idea.

Followed by the call:

IntPtr hwndMain = OpenPaint(); // Method that opens MSPaint

This way you should be fine getting the right windowhandle, and your code should be working, no matter how microsoft called it in win10

Swords

I've had it like this before but then it didn't work at all. When using Console.WriteLine(process.ProcessName); the output is "mspaint". The cursor moves inside paint but it doesn't click... – Cateran

But you don't ask for the process name - you're looking for the windowName, that's something different. If the lpWindowName parameter is not NULL, FindWindow calls the GetWindowText function to retrieve the window name for comparison See: msdn.microsoft.com/de-de/library/windows/desktop/… – Swords

@diiN_ maybe it would help if you provide the code of ` OpenPaint(); // Method that opens MSPaint` - maybe there is some error. Btw: did you debug to see if FindWindow gets handle back, or if it is zero? – Swords

@diiN_ Okay - I'll try this out - are you on Win7/8/10 ? – Swords

I'm using Win 10. – Cateran

@diiN_ okay - this might be why the name doesn't work - I'll edit my answer - got another (I think even better way to do this) – Swords

Thank you so much. The only thing I had to add to your solution was Thread.Sleep(500); before returning the handle. Without waiting the return value stayed 0. It works now. – Cateran

@diiN_ Fine I could help you! I added the Thread.Sleep() to my answer, for others passing by ;) – Swords

Answer 2

IntPtr hwndMain = FindWindow("mspaint", null);

That isn't good enough. Common mistake in pinvoke code, C# programmers tend to rely entirely too much on an exception to jump off the screen and slap them in the face to tell them that something went wrong. The .NET Framework does do that extra-ordinarily well. But that does not work the same way when you use an api that's based on the C language, like the winapi. C is a dinosaur language and did not support exceptions at all. It still doesn't. You'll only get an exception when the pinvoke plumbing failed, usually because of the bad [DllImport] declaration or a missing DLL. It does not speak up when the function executed successfully but returns a failure return code.

That does make it entirely your own job to detect and report failure. Just turn to the MSDN documentation, it always tells you how a winapi function indicates a mishap. Not completely consistent, so you do have to look, in this case FindWindow returns null when the window could not be found. So always code it like this:

IntPtr hwndMain = FindWindow("mspaint", null);
if (hwndMain == IntPtr.Zero) throw new System.ComponentModel.Win32Exception();

Do this for all the other pinvokes as well. Now you can get ahead, you'll reliably get an exception instead of plowing on with bad data. Which, as is so often the case with bad data, isn't quite bad enough. NULL is actually a valid window handle, the OS will assume you meant the desktop window. Ouch. You are automating the completely wrong process.

---

Understanding why FindWindow() fails does require a bit of insight, it is not very intuitive, but good error reporting is crucial to get there. The Process.Start() method only ensures that the program got started, it does not in any way wait until the process has completed its initialization. And in this case, it does not wait until it has created its main window. So the FindWindow() call executes about, oh, a couple of dozen milliseconds too early. Extra puzzling since it works just fine when you debug and single-step through the code.

Perhaps you recognize this kind of mishap, it is a threading race bug. The most dastardly kind of programming bug. Infamous for not causing failure consistently and very hard to debug since races are timing dependent.

Hopefully you realize that the proposed solution in the accepted answer is not good enough either. Arbitrarily adding Thread.Sleep(500) merely improves the odds that you now wait long enough before calling FindWindow(). But how do you know that 500 is good enough? It is always good enough?

No. Thread.Sleep() is never the correct solution for a threading race bug. If the user's machine is slow or is too heavily loaded with a shortage of available unmapped RAM then a couple of milliseconds turns into seconds. You have to deal with the worst case, and it is worst indeed, only ~10 seconds is in general the minimum you need to consider when the machine start thrashing. That's getting very unpractical.

Interlocking this reliably is such a common need that the OS has a heuristic for it. Needs to be a heuristic instead of a WaitOne() call on a synchronization object since the process itself doesn't cooperate at all. You can generally assume that a GUI program has progressed sufficiently when it starts asking for notifications. "Pumping the message loop" in Windows vernacular. That heuristic also made it into the Process class. Fix:

private static void OpenPaint()
{
    Process.process = new Process();
    process.StartInfo.FileName = "mspaint.exe";
    process.StartInfo.WindowStyle = "ProcessWindowStyle.Maximized;
    process.Start();
    process.WaitForInputIdle();          // <=== NOTE: added
}

I would be remiss if I didn't point out that you should use the built-in api for this. Called UI Automation, wrapped ably in the System.Windows.Automation namespace. Takes care of all those nasty little details, like threading races and turning error codes into good exceptions. Most relevant tutorial is probably here.

Mihe