CefSharp webpage element click
Asked Answered
Z

2

5

I'm trying to do simple click on some page element(like a btn or link).

I have wrote 2 functions for clicking via xpath and via CSS selectors.

Both of those functions perfectly works in browser's developer console, but partially does'nt work in CEF.

  • code perfectly click's in simple links from Developer Console and from Cef
  • code perfectly click's on exact button from Developer Console but doesn't do click from CEF. It's just ignore it for some reason...

How can this be? Js code is absolutely the same!...

    public void Click(string xpath)
    {
        var js = "document.evaluate(\"" + xpath + "\", document, null, XPathResult.ANY_TYPE, null).iterateNext().click();";

        EvaluateJavascript(js);
    }

    public void ClickCss(string css)
    {
        var js = "document.querySelector('"+ css + "').click()";

        EvaluateJavascript(js);
    }


    public async Task EvaluateJavascript(string script)
    {
        JavascriptResponse javascriptResponse = await Browser.GetMainFrame().EvaluateScriptAsync(script);

        if (!javascriptResponse.Success)
        {
            throw new JavascriptException(javascriptResponse.Message);
        }
    }

details: enter image description here

enter image description here

used click code:

_browser.ClickCss("#upload-container a");

one more time: same js code perfectly works in browser dev console, but doesn't work in CEF for some reason.

By the way, I have tested JS code in Chrome. So WebEngine is the same in both situations.

PS: Also will work for me simulation of drag-and-drop of some specific file to some specific web-element. But I didn't found any information about this not for Cef, not for Js, not for JQuery... =(

Zamudio answered 3/8, 2018 at 22:21 Comment(8)
Possibly blog.mariusschulz.com/2016/05/31/… is relevantDrachma
Try a simpler exampleDrachma
@Drachma but the same code works in browser. I didn't get "popup blocked" message in my browser. OpenFileDialog was successfully opened in Chrome and Firefox wit the same code and the same button.Zamudio
@Drachma on the simplier example (just a link) work's well in Cef. I have wrote this in my question :)Zamudio
CEF also has devtools, you can use them for debugging. If you are trying to open a file dialog then I believe your hitting a security restriction, see what messages appear in devtools.Drachma
I do not see any security messages on Security tab os DevTools. But I can sucessfully run code from it's console. Is it possible to run JS code from cef devtools programmatically? And by the way: BrSettings.WebSecurity = CefState.Disabled; on browser startZamudio
The simplest option is to set focus to the element and send an enter key press, you can probably adapt gist.github.com/jankurianski/… quite easilyDrachma
@Drachma thanks for idea, but... this button cannot be focused on the page for some reason. Even if I will try to make a focus manually, it's cannot be focused. :/ even without any code. Also I have tried code, it doesn't work anyway. But idea was great! :)Zamudio
Z
8

The problem was in security limitations of JS code.

Solution of the problem is:

  1. Get coordinates of a button/link with JS code
  2. Simulate click action on it with CEF:

    public void MouseClick(int x, int y)
    {
        Browser.GetBrowser().GetHost().SendMouseClickEvent(x, y, MouseButtonType.Left, false, 1, CefEventFlags.None);
        Thread.Sleep(15);
        Browser.GetBrowser().GetHost().SendMouseClickEvent(x, y, MouseButtonType.Left, true, 1, CefEventFlags.None);
    }
    
Zamudio answered 4/8, 2018 at 19:42 Comment(5)
Greatly appreciated, served as inspiration for KeyEvent issues I had. When JavaScript cannot, CefSharp saves the day.Shultz
How did you get the correct coordinates for SendMouseClickEvent?Aeroneurosis
that coordinates coming from the element right? i wonder what is the meaning of true / false of the same function call in that mouse click? Is that pressed in and released ?Egidio
@Egidio mouseUp cefsharp.github.io/api/57.0.0/html/…Zamudio
took me 3 days to get to this answer, didn't know about JS restrictions on firing things from code like the file selectorFlyblow
A
8

The answer above works ok. I just like to add some information that I think is very important...


Get DOM Element's position, and simulate a Mouse Click (and Move):

// get button's position
string jsonString = null;
var jsReponse = await chromiumWebBrowser1.EvaluateScriptAsync(
@"(function () {
    var bnt = document.getElementById('pnnext');
    bnt.focus();
    var bntRect = bnt.getBoundingClientRect();
    return JSON.stringify({ x: bntRect.left, y: bntRect.top });
})();"
);
if (jsReponse.Success && jsReponse.Result != null)
    jsonString = (string)jsReponse.Result;

// send mouse click event
if (jsonString != null)
{
    var jsonObject = JObject.Parse(jsonString);
    var xPosition = (int)jsonObject["x"] + 1;  // add +1 pixel to the click position
    var yPosition = (int)jsonObject["y"] + 1;  // add +1 pixel to the click position

    var host = chromiumWebBrowser1.GetBrowser().GetHost();
    host.SendMouseMoveEvent(xPosition, yPosition, false, CefEventFlags.None);
    Thread.Sleep(50);
    host.SendMouseClickEvent(xPosition, yPosition, MouseButtonType.Left, false, 1, CefEventFlags.None);
    Thread.Sleep(50);
    host.SendMouseClickEvent(xPosition, yPosition, MouseButtonType.Left, true, 1, CefEventFlags.None);
}

1 - The best way to get a DOM element's position on CefSharp, up to the present moment, is executing a custom JavaScript, and parse the result back. That's the purpose of the first segment of the code above. It gets an element by its ID (as defined on the HTML structure of the loaded web page), then gets its position relative to the viewport, and then returns it back to the C# code. Back on C#, we parse it using Newtonsoft's JObject.Parse method.

2 - The clicking occurs at the specified x and y coordinates, but is very important to know that this coordinates are relative to the browser component window (viewport). So if your web browser component is only 100px tall and you send a clicking command to y coordinate as 150px the clicking will occur but outside the web browser, hence, not in the web page.

3 - One trick to see if the click is really being executed, and where exactly, is change it from MouseButtonType.Left to MouseButtonType.Right. Doing it, if there is no restrictions on the web page, you will be able to see the "mouse right button menu". If you are clicking outside, chances are that you'll see a "mouse right button menu" on the OS (Windows here, and I was able to see that I was clicking outside my component with this trick) .

4 - Sometimes (specially with anti-crawler environments) it's necessary to also mimic the mouse movement. I do it using the method SendMouseMoveEvent as shown in the first line.

5 - If you have a very tall page loaded in your component, and you need to click on a button really outside the boundaries of the component, you need to scroll the web page. You can do it execution some JavaScript as on the example code above, or You can do it with code below:


Simulate a Mouse Scroll (Mouse Wheel Event):

// send scroll command
// (mouse position X, mouse position Y, how much to scroll X, how much to scroll Y, events)
chromiumWebBrowser1.GetBrowser().GetHost().SendMouseWheelEvent(10, 10, 0, -100, CefEventFlags.None); 
Thread.Sleep(300);

The biggest part of the parameters are self-explanatory, but the most important part here is the 3rd and 4th parameters, which are respectively "how much to scroll X" and "how much to scroll Y". To scroll down, just use negative values, and to scroll up, positive ones. In the code above, it scrolls nothing (zero pixels) horizontally (x axis) and 100px down vertically (y axis).

You can use this code inside a loop to scroll how much you need. I use it in conjunction with a JavaScript code to retrieve the position of a button, to detect if it's on the viewport, or if i need to send a scroll event again.

Agrostology answered 6/12, 2020 at 2:14 Comment(5)
Using JSON to represent x,y coordinates seems overkill and has unnessicary overhead. Just return the two values in an array in JavaScript and in .Net you'll get a corresponding array with a simple cast. Mixing await and ContinueWith is a little clunky, using one or another would be better from a code readability point of view.Drachma
@Drachma thanks for your revision. About "using JSON is an overkill", yeah but it keeps everything tidy, and if for any reason I change the order of parameters in JS or need to return some other values, I don't need to change the rest of the code. I think it's too little processing overhead to have a great way to keep data organized. If I use a simple array, I won't have any way to name the properties, and will rely only on the order of elements in the array, which could be a maintenance nightmare...Agrostology
@Drachma about the "mixing await and ContinueWith" you're totally right. It was bad design, thanks for pointing it up. I rewrite it, and think it's way better now. Let me know what do you think.Agrostology
If you wish to name the properties then return a simple object in JavaScript and you'll get a dictionary in .Net with the key value pairs. Await usage is much easier to read now :+1:Drachma
Regarding #5 (scrolling the page to make the element visible) in my tests, sometimes, depending on the context, executing in JS element.focus(); (as you have it in your JS code) does that already - scrolls the page to that element to make it visible.Pyx

© 2022 - 2024 — McMap. All rights reserved.