What's the difference between `driver.execute_script("...")` and `driver.get("javascript: ..."` with geckodriver/Firefox?

Question

I think, this question concerns the internal workings of Selenium. In another post Referer missing in HTTP header of Selenium request it becomes apparent that there is a difference between running

driver.execute_script("window.location.href = '{}';".format(url))

and

driver.get("javascript: window.location.href = '{}'".format(url))

The latter command will send Referer header with the request, the former will not.

Doesn't matter at this point if this is desired behavior or a bug and Referer should be sent by both commands. Also, window.location.href = ... is only an example.

Yet, obviously, there must be a difference between running JavaScript with command driver.execute_script("...") and driver.get("javascript: ..." if they don't produce the same result. So the question is more about the fact that both commands don't invoke the same Selenium code internally.

What is the difference between both commands?

Flog

Why do you want to invoke an url in a complex manner as javascript: window.location.href = '{}'".format(url) where you have the much proven and robust get("{}".format(url))? What is your exact usecase? – Rosina

it's probably the same call, which means the .get example calls (window.location.href = "") twice. Execute_script will also inject first, then make the call. – Gerry

Other differences would involve whatever validation/utility methods are inside of .get. (It may clean the url a bit... not sure what it does when there's a javascript call inside of it... but it could possibly be replacing the whole thing with just the URL so that it doesn't execute twice or get into some kind of loop.) – Gerry

I was assuming Selenium was using window.location.href for the page change. Now that I looked at the source, that is not the case. (Smacks head... of course because the first page upon launching the browser is blank and JS can't be inserted!) .get (or navigateto) actually sends a get request. So the difference is really that when you use the .get method the in-between server (proxy server? node?) is sending the get request, and not the browser. So the proxy has no referrer at that point but your local call. – Gerry

Answer 1

The answer to your question depends on the browser your driver is running. Selenium itself does not implement these functionalities - it merely invokes the underlying driver's API.

Take a look at the source of WebDriver.execute_script and WebDriver.get - they both just call self.execute, which performs a request to the webdriver.

Chrome, for example, does not support 'javascript:' urls with WebDriver.get since 2013, as can bee seen in chromium's webdriver implementation.

The actual difference between directly running a JS script and navigating to a 'javascript URL' is embedded deep in each browser's implementation, and might be not very straightforward. A possible reason for the difference you mentioned could be an implementation detail - maybe the browser (that was used when the results you mentioned were produced) only sends a Referer header when it is in the context of a high level navigation command (driver.get), and therefore did not include one on a plain javascript-triggered navigation.

Gravitation

According to what you wrote there, it does seem that my guess is wrong or incomplete. But looking at the geckodriver code, I didn't see any mention of JS URLs, which leads me to think all relevant code is inside the firefox engine, and not selenium or even geckodriver. – Gravitation

Is there anything I should add to my answer? – Gravitation

Answer 2

TL;DR: I was curious about this and was starting an answer. Then went out of town.

I'm not trying to poach points or anything from @Ni. As he points out, get and execute_script call self.execute, which--in turn--calls a method from the Command class. For example, Command.GET or Command.EXECUTE_SCRIPT. And that's where the trail went cold for me...

---

the source code

https://github.com/SeleniumHQ/selenium/blob/master/py/selenium/webdriver/remote/webdriver.py

def get(self, url):
    """
    Loads a web page in the current browser session.
    """
    self.execute(Command.GET, {'url': url})

and

def execute_script(self, script, *args):
    """
    Synchronously Executes JavaScript in the current window/frame.

    :Args:
     - script: The JavaScript to execute.
     - \*args: Any applicable arguments for your JavaScript.

    :Usage:
        driver.execute_script('return document.title;')
    """
    converted_args = list(args)
    command = None
    if self.w3c:
        command = Command.W3C_EXECUTE_SCRIPT
    else:
        command = Command.EXECUTE_SCRIPT

    return self.execute(command, {
        'script': script,
        'args': converted_args})['value']

which points to

https://github.com/SeleniumHQ/selenium/blob/master/py/selenium/webdriver/remote/command.py

class Command(object):
    """
    Defines constants for the standard WebDriver commands.
    While these constants have no meaning in and of themselves, they are
    used to marshal commands through a service that implements WebDriver's
    remote wire protocol:
        https://github.com/SeleniumHQ/selenium/wiki/JsonWireProtocol
    """

and

https://github.com/SeleniumHQ/selenium/blob/master/py/selenium/webdriver/remote/remote_connection.py#L142 shows a private method called self._commands, which is a dictionary containing commands that mirror the syntax seen in ..remote/webdriver.py

For example: Command.GET: ('POST', '/session/$sessionId/url') vs. self.execute(Command.GET, {'url': url})

The endpoints in self._commands correspond to the https://github.com/SeleniumHQ/selenium/wiki/JsonWireProtocol#command-reference, so this is the service "used to marshal commands" (?) or part of it...

(ruby equiv: https://github.com/SeleniumHQ/selenium/blob/master/rb/lib/selenium/webdriver/remote/commands.rb)

Madelynmademoiselle

@finefoot: sorry, I don't. I'm curious how the commands are sent under-the-hood, so I dug a bit. Still digging :) – Madelynmademoiselle