selenium.common.exceptions.InvalidCookieDomainException: Message: invalid cookie domain while executing tests in Django with Selenium

Question

I am setting up tests for both chrome and firefox using seleniumgrid.I am using docker images selenium-hub and selenium node-chrome and node-firefox as below.

  app:
    build: .
    command: gunicorn --reload --capture-output --log-level debug --access-logfile - -w 3 -b 0.0.0.0 app.wsgi
    restart: always
    volumes_from:
        - initialize
    ports:
      - "8000:8000"
    links:
      - db
      - rabbitmq
      - selenium_hub
    env_file: secrets.env
    volumes:
        - ./app/:/code/

  selenium_hub:
    image: selenium/hub
    ports:
      - 4444:4444
    expose:
      - 4444
    tty: true
    environment:
      - GRID_MAX_SESSION=20
      - GRID_NEW_SESSION_WAIT_TIMEOUT=60000
      - GRID_BROWSER_TIMEOUT=300
      - GRID_TIMEOUT=300
      - TIMEOUT=300
  node_1:
    image: selenium/node-chrome
    depends_on:
      - selenium_hub
    environment:
      - HUB_HOST=selenium_hub
      - HUB_PORT=4444
      - NODE_MAX_SESSION=3
      - NODE_MAX_INSTANCES=2
    shm_size: 2GB
  node_2:
    image: selenium/node-firefox
    environment:
      - HUB_HOST=selenium_hub
      - HUB_PORT=4444
      - NODE_MAX_SESSION=3
      - NODE_MAX_INSTANCES=2
    shm_size: 2GB
    depends_on:
      - selenium_hub

When I try to run the tests I am always running into this error InvalidCookieDomainException: Message: invalid cookie domain. I have already set domain to self.live_server_url. below is the full traceback with the test setup.

  Traceback (most recent call last):
    File "/code/frontend/tests/test_user_auth.py", line 75, in setUp
        "port": "8082",
    File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 894, in add_cookie
        self.execute(Command.ADD_COOKIE, {'cookie': cookie_dict})
    File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
        self.error_handler.check_response(response)
    File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
        raise exception_class(message, screen, stacktrace)
    selenium.common.exceptions.InvalidCookieDomainException: Message: invalid cookie domain
    (Session info: chrome=77.0.3865.75)

Test reference tutorial.

class TestUserCreate(StaticLiveServerTestCase):
    fixtures = ["test.json"]
    port = 8082

    @classmethod
    def setUpClass(cls):
        super().setUpClass()
        caps = {
            "browserName": os.getenv("BROWSER", "chrome"),
            "javascriptEnabled": True,
         }
        cls.driver = webdriver.Remote(
            command_executor="http://selenium_hub:4444/wd/hub",
            desired_capabilities=caps,
         )
        cls.driver.implicitly_wait(10)

    @classmethod
    def tearDownClass(cls):
        cls.driver.quit()
        super().tearDownClass()

    def setUp(self):
        # Login the user
        self.assertTrue(self.client.login(username="james", password="changemequick"))

        # Add cookie to log in the browser
        cookie = self.client.cookies["sessionid"]
        self.driver.get(self.live_server_url + reverse("find_patient"))
        self.driver.add_cookie(
            {
                "name": "sessionid",
                "value": cookie.value,
                "domain": "localhost"
            }
         )
        super().setUp()

    def test_form_loader(self):
        # test forms loader is functioning properly

        driver = self.driver
        driver.get(self.live_server_url + "/accounts/login/")

        driver.find_element_by_xpath("//input[@type='submit']").click()
        driver.get_screenshot_as_file("login.png")
        assert len(driver.find_elements_by_css_selector(".loading")) == 0

Mincing

see also How to preload cookies before first request with Python3, Selenium Chrome WebDriver? – Derogative

Answer 1

In selenium we must use the URL of the testing server by default this is localhost, to enable access to an external accessible server address which is the hosting docker container's address we need to use the server machine's ip address, So set as below,

class BaseTestCase(StaticLiveServerTestCase):
"""
Provides base test class which connects to the Docker
container running Selenium.
"""
host = '0.0.0.0'  # Bind to 0.0.0.0 to allow external access

@classmethod
def setUpClass(cls):
    super().setUpClass()
    # Set host to externally accessible web server address
    cls.host = socket.gethostbyname(socket.gethostname())

reference here

Mincing

Answer 2

This error message...

selenium.common.exceptions.InvalidCookieDomainException: Message: invalid cookie domain

...implies that an illegal attempt was made to set a cookie under a different domain than that of the current document.

---

Details

As per the HTML-Living Standard Specs a Document Object may be categorized as a cookie-averse Document object in the following circumstances :

• A Document that has no Browsing Context.
• A Document whose URL's scheme is not a network scheme.

---

Deep Dive

As per Invalid cookie domain this error may occur if the current domain were to be example.com, it would not be possible to add the cookie for the domain example.org.

As an example:

• Sample Code:

  from selenium import webdriver
  from selenium.common import exceptions
  
  session = webdriver.Firefox()
  session.get("https://example.com/")
  try:
      cookie = {"name": "foo",
            "value": "bar",
            "domain": "example.org"}
      session.add_cookie(cookie)
  except exceptions.InvalidCookieDomainException as e:
      print(e.message)
  

• Console Output:

  InvalidCookieDomainException: https://example.org/
  

---

Solution

If you have stored the cookie from domain example.com, these stored cookies can't be pushed through the webdriver session to any other different domanin e.g. example.edu. The stored cookies can be used only within example.com. Further, to automatically login an user in future, you need to store the cookies only once, and that's when the user have logged in. Before adding back the cookies you need to browse to the same domain from where the cookies were collected.

---

Example

As an example, you can store the cookies once the user havd logged in within an application as follows:

from selenium import webdriver
import pickle

driver = webdriver.Chrome()
driver.get('http://demo.guru99.com/test/cookie/selenium_aut.php')
driver.find_element_by_name("username").send_keys("abc123")
driver.find_element_by_name("password").send_keys("123xyz")
driver.find_element_by_name("submit").click()

# storing the cookies
pickle.dump( driver.get_cookies() , open("cookies.pkl","wb"))
driver.quit()

Later if you want the user automatically logged-in, you need to browse to the specific domain /url first and then you have to add the cookies as follows:

from selenium import webdriver
import pickle

driver = webdriver.Chrome()
driver.get('http://demo.guru99.com/test/cookie/selenium_aut.php')

# loading the stored cookies
cookies = pickle.load(open("cookies.pkl", "rb"))
for cookie in cookies:
    # adding the cookies to the session through webdriver instance
    driver.add_cookie(cookie)
driver.get('http://demo.guru99.com/test/cookie/selenium_cookie.php')

---

Additional Consideration

It seems you are using chrome=77.0.3865.75. Ideally you need to ensure that:

• Selenium is upgraded to current levels Version 3.141.59.
• ChromeDriver is updated to current ChromeDriver v79.0.3945.36 level.
• Chrome is updated to current Chrome Version 79.0 level. (as per ChromeDriver v79.0 release notes)

---

Reference

You can find a detailed discussion in:

• Error when loading cookies into a Python request session

Acre

Will the cookies work also for a different browser? – Mincing

@Mincing This did not work for me using FireFox (geckodriver). – Enterotomy

@Mincing Neither the parent answer to this comment nor the accepted answer worked for me with FireFox. – Enterotomy

@CameronJewell are you on docker, or provide more info through a question. – Mincing

@Mincing Not asking a question; just providing my experience as a response to yours – Enterotomy

You are perfect! And by the way, you don't need to go back to the specific url with which you got the cookies from. Although I didn't use pickle, I copied the cookies the hard way. I believe it's the same all round. Just reload the website after adding cookies and that should work just fine. – Painless

Answer 3

I run into a similar problem while working with C#. I solved it by first setting the URL to the home page that does not need a user session and then setting the cookies and after that opening the page requiring a session. Here is the code

webDriver = new ChromeDriver(".");
        String f = @"./scrapper/cookies.data";
        String[] lines = File.ReadAllLines(f);
        webDriver.Url = "https://myhomepage.com/";
        foreach (String ln in lines)
        {
            String[] temp = ln.Split(';');
            String name = temp[0];
            String value = temp[1];
            String domain = temp[2];
            String path = temp[3];
            DateTime expiry = DateTime.Now.AddDays(60);
            if (!String.IsNullOrEmpty(temp[4]))
            {
                expiry = DateTime.Parse(temp[4]);
            }
            Boolean isSecure = Boolean.Parse(temp[5]);
            Cookie cookie = new Cookie(name, value, domain, path, expiry);

            webDriver.Manage().Cookies.AddCookie(cookie);
        }
        webDriver.Url = "https://target_page_requiring_session.php";

        IWebElement tab3 = webDriver.FindElement(By.XPath("//*[@id='TabIDS_3']"));
        tab3.Click();

Microlith

Answer 4

I run into the same problem. My solution was simply to avoid setting domain value (wich is optional, by the way) in the cookie.

    class MyTestCase(StaticLiveServerTestCase):
        host = os.environ['DJANGO_CONTAINER_NAME']

        def test_logged_in(self):
            self.client.login(username="integration", password="test")
            self.selenium.get(self.live_server_url)
            self.selenium.add_cookie({"name": settings.SESSION_COOKIE_NAME, 
                                    "value": self.client.cookies[settings.SESSION_COOKIE_NAME].value})
            post = Post.objects.first()
            self.selenium.get(f"{self.live_server_url}{reverse('post', args=(post.slug, ))}")

The main point here is to let the Django client (here self.client) login and then copy its session cookie to a Selenium browser session, which should have already visited a page from your live server - in other words, first let the Selenium browser open a page, then set the cookie (skipping domain) and then, the selenium browser is logged in.

Leboff