I'm using selenium python webdriver in order to browse some pages. I want to inject a javascript code in to a pages before any other Javascript codes get loaded and executed. On the other hand, I need my JS code to be executed as the first JS code of that page. Is there a way to do that by Selenium?
I googled it for a couple of hours, but I couldn't find any proper answer!
If you cannot modify the page content, you may use a proxy, or use a content script in an extension installed in your browser. Doing it within selenium you would write some code that injects the script as one of the children of an existing element, but you won't be able to have it run before the page is loaded (when your driver's get() call returns.)
String name = (String) ((JavascriptExecutor) driver).executeScript(
"(function () { ... })();" ...
The documentation leaves unspecified the moment at which the code would start executing. You would want it to before the DOM starts loading so that guarantee might only be satisfiable with the proxy or extension content script route.
If you can instrument your page with a minimal harness, you may detect the presence of a special url query parameter and load additional content, but you need to do so using an inline script. Pseudocode:
<html>
<head>
<script type="text/javascript">
(function () {
if (location && location.href && location.href.indexOf("SELENIUM_TEST") >= 0) {
var injectScript = document.createElement("script");
injectScript.setAttribute("type", "text/javascript");
//another option is to perform a synchronous XHR and inject via innerText.
injectScript.setAttribute("src", URL_OF_EXTRA_SCRIPT);
document.documentElement.appendChild(injectScript);
//optional. cleaner to remove. it has already been loaded at this point.
document.documentElement.removeChild(injectScript);
}
})();
</script>
...
executeScript call completes successfully, but the function I'm trying to inject doesn't appear to persist (typeof myFunction == 'undefined'). If I run the same code directly in Developer Tools console, however, I get the expected result (typeof myFunction == 'function'). Do you have any suggestions for diagnosing this issue? –
Appurtenance Selenium has now supported Chrome Devtools Protocol (CDP) API, so , it is really easy to execute a script on every page load. Here is an example code for that:
driver.execute_cdp_cmd('Page.addScriptToEvaluateOnNewDocument', {'source': 'alert("Hooray! I did it!")'})
And it will execute that script for EVERY page load. More information about this can be found at:
Since version 1.0.9, selenium-wire has gained the functionality to modify responses to requests. Below is an example of this functionality to inject a script into a page before it reaches a webbrowser.
import os
from seleniumwire import webdriver
from gzip import compress, decompress
from urllib.parse import urlparse
from lxml import html
from lxml.etree import ParserError
from lxml.html import builder
script_elem_to_inject = builder.SCRIPT('alert("injected")')
def inject(req, req_body, res, res_body):
# various checks to make sure we're only injecting the script on appropriate responses
# we check that the content type is HTML, that the status code is 200, and that the encoding is gzip
if res.headers.get_content_subtype() != 'html' or res.status != 200 or res.getheader('Content-Encoding') != 'gzip':
return None
try:
parsed_html = html.fromstring(decompress(res_body))
except ParserError:
return None
try:
parsed_html.head.insert(0, script_elem_to_inject)
except IndexError: # no head element
return None
return compress(html.tostring(parsed_html))
drv = webdriver.Firefox(seleniumwire_options={'custom_response_handler': inject})
drv.header_overrides = {'Accept-Encoding': 'gzip'} # ensure we only get gzip encoded responses
Another way in general to control a browser remotely and be able to inject a script before the pages content loads would be to use a library based on a separate protocol entirely, eg: Chrome DevTools Protocol. The most fully featured I know of is playwright
injected.append((req, req_body, res, res_body, parsed_html))? I didn't find what injected refers to –
Pastelist custom_response_handler injection function allows to modify the response headers? I see we can return the response body, but in my example I would also want to add or modify a header in the response. –
Pastelist res.headers. –
Nigro If you want to inject something into the html of a page before it gets parsed and executed by the browser I would suggest that you use a proxy such as Mitmproxy.
If you cannot modify the page content, you may use a proxy, or use a content script in an extension installed in your browser. Doing it within selenium you would write some code that injects the script as one of the children of an existing element, but you won't be able to have it run before the page is loaded (when your driver's get() call returns.)
String name = (String) ((JavascriptExecutor) driver).executeScript(
"(function () { ... })();" ...
The documentation leaves unspecified the moment at which the code would start executing. You would want it to before the DOM starts loading so that guarantee might only be satisfiable with the proxy or extension content script route.
If you can instrument your page with a minimal harness, you may detect the presence of a special url query parameter and load additional content, but you need to do so using an inline script. Pseudocode:
<html>
<head>
<script type="text/javascript">
(function () {
if (location && location.href && location.href.indexOf("SELENIUM_TEST") >= 0) {
var injectScript = document.createElement("script");
injectScript.setAttribute("type", "text/javascript");
//another option is to perform a synchronous XHR and inject via innerText.
injectScript.setAttribute("src", URL_OF_EXTRA_SCRIPT);
document.documentElement.appendChild(injectScript);
//optional. cleaner to remove. it has already been loaded at this point.
document.documentElement.removeChild(injectScript);
}
})();
</script>
...
executeScript call completes successfully, but the function I'm trying to inject doesn't appear to persist (typeof myFunction == 'undefined'). If I run the same code directly in Developer Tools console, however, I get the expected result (typeof myFunction == 'function'). Do you have any suggestions for diagnosing this issue? –
Appurtenance so I know it's been a few years, but I've found a way to do this without modifying the webpage's content and without using a proxy! I'm using the nodejs version, but presumably the API is consistent for other languages as well. What you want to do is as follows
const {Builder, By, Key, until, Capabilities} = require('selenium-webdriver');
const capabilities = new Capabilities();
capabilities.setPageLoadStrategy('eager'); // Options are 'eager', 'none', 'normal'
let driver = await new Builder().forBrowser('firefox').setFirefoxOptions(capabilities).build();
await driver.get('http://example.com');
driver.executeScript(\`
console.log('hello'
\`)
That 'eager' option works for me. You may need to use the 'none' option. Documentation: https://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/lib/capabilities_exports_PageLoadStrategy.html
EDIT: Note that the 'eager' option has not been implemented in Chrome yet...
Updated version of @Mattwmaster58's answer that works with the latest version of selenium-wire (5.1.0 at the time of this writing). Also adds support for nonce attributes on inline script tags.
from lxml import html
from lxml.etree import ParserError
from lxml.html import builder
from seleniumwire import webdriver
from seleniumwire.request import Request, Response
from seleniumwire.thirdparty.mitmproxy.net.http import encoding as decoder
SCRIPT_BODY_TO_INJECT = 'alert("injected")'
def has_mime_type(header: str, expected_type: str) -> bool:
return header == expected_type or header.startswith(expected_type + ";")
def response_interceptor(request: Request, response: Response) -> None:
content_type = response.headers.get("Content-Type")
if (
response.status_code != 200
or not content_type
or not has_mime_type(content_type, "text/html")
):
return
encoding = response.headers.get("Content-Encoding", "identity")
try:
parsed_html = html.fromstring(decoder.decode(response.body, encoding))
except ParserError:
return
# Preserve nonce attribute to allow inline script.
# https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/nonce
attrs = {}
if (nonce_script := parsed_html.find(".//script[@nonce]")) is not None:
attrs["nonce"] = nonce_script.get("nonce")
try:
injected_script = builder.SCRIPT(SCRIPT_BODY_TO_INJECT, **attrs)
parsed_html.head.insert(0, injected_script)
except IndexError: # No head element.
return
response.body = decoder.encode(
html.tostring(parsed_html.getroottree()), encoding
)
del response.headers["Content-Length"] # Avoid duplicate header.
response.headers["Content-Length"] = str(len(response.body))
def main():
with webdriver.Firefox() as session:
session.response_interceptor = response_interceptor
session.get("https://example.com")
if __name__ == "__main__":
main()
As an alternative to generating output with lxml (which can alter the structure of the HTML), you can also use a regex to insert the tag and preserve existing formatting:
from lxml import html
from lxml.etree import ParserError
from lxml.html import builder
from mimeparse import parse_mime_type
from seleniumwire import webdriver
from seleniumwire.request import Request, Response
from seleniumwire.thirdparty.mitmproxy.net.http import encoding as decoder
import re
SCRIPT_BODY_TO_INJECT = 'alert("injected")'
HEAD_TAG_RE = re.compile(r"<head\s*>()", re.IGNORECASE)
INLINE_SCRIPT_TAG_RE = re.compile(
r"()<script\b(?:(?!\bsrc\b\s*=\s*['\"]).)*?>", re.IGNORECASE
)
def response_interceptor(request: Request, response: Response) -> None:
content_type = response.headers.get("content-type")
if not content_type:
return
mime_type, mime_subtype, mime_params = parse_mime_type(content_type)
if (
response.status_code != 200
or mime_type != "text"
or mime_subtype != "html"
):
return
encoding = response.headers.get("content-encoding", "identity")
charset = mime_params.get("charset", "iso-8859-1")
try:
decoded_body = decoder.decode(response.body, encoding).decode(charset)
parsed_html = html.fromstring(decoded_body)
except ParserError:
return
# Preserve nonce attribute to allow inline script.
# https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/nonce
attrs = {}
if (nonce_script := parsed_html.find(".//script[@nonce]")) is not None:
attrs["nonce"] = nonce_script.get("nonce")
# Some sites inject scripts before the DOCTYPE, which isn't valid markup
# but still runs.
if m := min((x for regex in (INLINE_SCRIPT_TAG_RE, HEAD_TAG_RE)
if (x := regex.search(decoded_body))),
key=lambda x: x.start()):
injected_script_text = html.tostring(
builder.SCRIPT(SCRIPT_BODY_TO_INJECT, **attrs), encoding="unicode"
)
replacement = (
m.string[m.start(): m.start(1)]
+ injected_script_text
+ m.string[m.start(1): m.end()]
)
modified_body = m.string[:m.start()] + replacement + m.string[m.end():]
response.body = decoder.encode(modified_body.encode(charset), encoding)
del response.headers["Content-Length"] # Avoid duplicate header.
response.headers["Content-Length"] = str(len(response.body))
def main():
with webdriver.Firefox() as session:
session.response_interceptor = response_interceptor
session.get("https://example.com")
if __name__ == "__main__":
main()
© 2022 - 2024 — McMap. All rights reserved.
onLoadtrigger to execute the script on page load. This is one of the most straightforward ways as I see it happening. – Chun