How do I read the local storage of my Safari Web Extension using Selenium in Python?

Question

With the Firefox WebDriver I can read the local storage of my extension like so:

extension_path = "/path/to/my/extension"
info = {
    "extension_id": f"foobar",
    "uuid": uuid.uuid4(),
}

base_url = f"moz-extension://{info['uuid']}/"

opts = FirefoxOptions()
opts.set_preference('extensions.webextensions.uuids', '{"%s": "%s"}' % (
    info["extension_id"], info["uuid"]))
driver = webdriver.Firefox(options=opts)
driver.install_addon(extension_path, temporary=True)

driver.get(f"{base_url}_generated_background_page.html")
results = self.driver.execute_async_script((
    "let done = arguments[arguments.length - 1],"
    "  store_name = arguments[0];"
    "browser.storage.local.get([store_name], function (res) {"
    "  done(res[store_name]);"
    "});"
), "foo")

How can I do the same with the Safari WebDriver on macOS? I've ported the extension using xcrun safari-web-extension-converter /path/to/my/extension and built and manually tested that it works in Safari. In Safari I can go to Develop -> Web Extension Background Pages -> <my web extension> to find the id of the extension and see that a generated background page is located at safari-web-extension://<id>/_generated_background_page.html

But running the following results in Selenium freezing at driver.get(f"{base_url}_generated_background_page.html")

base_url = f"safari-web-extension://<id>/"

driver = webdriver.Safari()
driver.get(f"{base_url}_generated_background_page.html")
results = self.driver.execute_async_script((
    "let done = arguments[arguments.length - 1],"
    "  store_name = arguments[0];"
    "browser.storage.local.get([store_name], function (res) {"
    "  done(res[store_name]);"
    "});"
), "foo")

What can I do?

Update Feb 8th 2023

I have also tried an approach using browser.runtime.sendMessage where in Python Selenium I do this:

results = self.driver.execute_async_script((
    "let done = arguments[arguments.length - 1],"
    "  store_name = arguments[0];"
    "  browser.runtime.sendMessage('com.oskar.foo.Extension (Apple Team ID)', {}, function (res) {"
    "    done(res[store_name]);"
    "  });"
), "foo")

and add the following to background.js in the extension:

browser.runtime.onMessageExternal.addListener(function (
  request,
  sender,
  sendResponse
) {
  browser.storage.local.get("foo").then((j) => {
    sendResponse(j);
  });
  return true;
});

and this to the manifest.json

"externally_connectable": {
    "ids": ["*"],
    "matches": ["https://example.org/*"]
}

This way I actually get a value from the extension when running the test. But instead of reading the storage of the extension from the Safari instance started by Selenium, it reads the storage of the extension from the "real" safari instance.

Wellbeloved

Safari doesn't allow unsigned web extensions, and because Selenium usually runs its own browser processes, it might not work as expected in Selenium even though it works on your local Safari. This link may help developer.apple.com/documentation/safariservices/… – Jacynth

@GuiLeFlea It does if you check "Allow Unsigned Extensions" before running Selenium. I can verify with another custom extension that adds a red border to every website, that even instances/windows created by Selenium has the unsigned extension activated – Wellbeloved

I believe the clues explaining why this isn't working are here.developer.apple.com/documentation/webkit/… To support WebDriver without sacrificing a user’s privacy or security, Safari’s driver provides extra safeguards to ensure that test execution is isolated from normal browsing data and from other test runs. – Jacynth

Is it possible that there is a difference in execution context between the WebDriver version of the extension and the local version? – Jacynth

maybe try the popup url – Firearm

Answer 1

I'm not very knowledgeable on the subject. However, such requests; It may be that it needs to provide duplex communication, including outgoing request and callback, and in this case, it may hang due to network needs.

In the link below, the transaction was made by polling with wait. I think this kind of approach can solve the problem. Also mentioned here is a bug, care it.

https://mcmap.net/q/374465/-understanding-execute-async-script-in-selenium

Mathewmathews

I've tried both execute_async_script and execute_script and neither works. The thing is that it seems to freeze on the line before, i.e. driver.get(f"{base_url}_generated_background_page.html") – Wellbeloved

According to this information, there seems to be something wrong with this code: driver.get expects a path for the service will run there. Please look that: selenium.dev/selenium/docs/api/py/webdriver_safari/… – Mathewmathews

In what way is that link relevant for driver.get? – Wellbeloved

I understanding from the document that: webdriver's safari instance's get method's current parameter is have to be a path not a file – Mathewmathews

Could you please provide details on how you would apply this to my problem in order to solve the hanging of driver.get(f"{base_url}_generated_background_page.html") ? – Wellbeloved