How to save and load cookies using Python + Selenium WebDriver

Asked 25/2, 2013 at 0:33 Answered 21/9, 2023 at 16:49

Solved python python-2.7 selenium webdriver

210

How can I save all cookies in Python's Selenium WebDriver to a .txt file, and then load them later?

The documentation doesn't say much of anything about the getCookies function.

Cowpoke answered 25/2, 2013 at 0:33 Comment(1)

Related question: the top answer below requires loading the page first before setting the cookie. Refer to python 3.x - How to preload cookies before first request with Python3, Selenium Chrome WebDriver? - Stack Overflow and python 3.x - Add cookies before first request in Python3 with Selenium Firefox Webdriver - Stack Overflow for alternatives. – Audiology 12/11, 2023 at 4:28

311

You can save the current cookies as a Python object using pickle. For example:

import pickle
import selenium.webdriver

driver = selenium.webdriver.Firefox()
driver.get("http://www.google.com")
pickle.dump(driver.get_cookies(), open("cookies.pkl", "wb"))

And later to add them back:

import pickle
import selenium.webdriver

driver = selenium.webdriver.Firefox()
driver.get("http://www.google.com")
cookies = pickle.load(open("cookies.pkl", "rb"))
for cookie in cookies:
    driver.add_cookie(cookie)

Orthodoxy answered 25/2, 2013 at 0:41 Comment(13)

will driver.add_cookie(cookie) replace the cookie in the loop and finally it can only add one cookie entry? – Manner 18/7, 2016 at 9:14

How can I get secure cookies? – Maggio 29/11, 2017 at 13:37

I get a recursion error if I copy this, marking down – Caesarea 20/12, 2018 at 11:8

Works perfectly fine on python 3.6.7. Saves authentication from previous session perfectly. No installation for 'pickle' required. – Schauer 3/6, 2019 at 4:58

I have an issue with this. It works fine however when I try to drive.add_cookie t again I got an error message saying "expiry" key is not valid. I am using chromedriver on Mac OS – Markus 12/8, 2019 at 22:32

might wanna call driver.delete_all_cookies() before loading the cookies. My code wasn't working without it. – Enkindle 15/12, 2019 at 2:0

I'm also getting selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: invalid 'expiry' – Dianetics 3/4, 2020 at 5:44

For those getting the error saying invalid 'expiry', see my fix here: https://mcmap.net/q/66894/-how-to-fix-quot-invalid-argument-invalid-39-expiry-39-quot-in-selenium-when-adding-cookies-to-a-chromedriver – Lighterman 17/5, 2020 at 1:12

Hi, I've used your code to write complete solution of loading cookies on starting browser and storing cookies in closing browser. It supports multi instances which is not possible via using user-data-dir. dev.to/hardiksondagar/… – Finbur 28/7, 2020 at 13:2

If you save the cookies as .JSON it is nice for reading and adjustments as well as easy to parse back into python – Kiss 23/12, 2020 at 12:10

What is the method to save/load JSON cookies?? – Coulisse 9/4, 2021 at 21:15

with this I did not able to login again with the previous saved cookies. – Arkwright 25/8, 2021 at 4:36

Websites have too many cookies. Its almost impossible to add cookies manually to driver object via json data. This method is very awesome and pratic. I log in manually at once and save cookies for future uses. – Kopans 5/11, 2022 at 14:27

153

When you need cookies from session to session, there is another way to do it. Use the Chrome options user-data-dir in order to use folders as profiles. I run:

# You need to: from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("user-data-dir=selenium")
driver = webdriver.Chrome(options=chrome_options)
#for selenium 4.15.2 options instead of chrome_options
#driver = webdriver.Chrome(options=chrome_options) 
driver.get("www.google.com")

Here you can do the logins that check for human interaction. I do this and then the cookies I need now every time I start the Webdriver with that folder everything is in there. You can also manually install the Extensions and have them in every session.

The second time I run, all the cookies are there:

# You need to: from selenium.webdriver.chrome.options import Options    
chrome_options = Options()
chrome_options.add_argument("user-data-dir=selenium") 
driver = webdriver.Chrome(options=chrome_options)
#for selenium 4.15.2 options instead of chrome_options
#driver = webdriver.Chrome(options=chrome_options) 
driver.get("www.google.com") # Now you can see the cookies, the settings, extensions, etc., and the logins done in the previous session are present here.

The advantage is you can use multiple folders with different settings and cookies, Extensions without the need to load, unload cookies, install and uninstall Extensions, change settings, change logins via code, and thus no way to have the logic of the program break, etc.

Also, this is faster than having to do it all by code.

Torbernite answered 7/2, 2018 at 13:52 Comment(11)

This was the best solution for me when dealing with Google logins. At some point my development usage was flagged as suspicious activity. – Plainclothesman 13/3, 2019 at 5:0

@p1g1n was flagged before or after using this solution – Torbernite 13/3, 2019 at 7:55

Sorry, it was flagged before using the solution. Now I stay logged in so there's no suspicious activity. – Plainclothesman 14/3, 2019 at 1:45

The code is a bit outdated, and is missing a required import. I will post a new answer. – Schauer 3/6, 2019 at 5:22

chrome_options = Options() gives me name 'Options' is not defined ... ? – Dianetics 3/4, 2020 at 5:37

@Dan you need to: from selenium.webdriver.chrome.options import Options – Torbernite 3/4, 2020 at 8:31

Doesnt seem to work with headless mode though but works! – Python 22/4, 2020 at 14:6

ERROR: * InvalidArgumentException: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir* – Indohittite 1/8, 2022 at 15:0

Make sure you use the entre path not just "selenium" or Chrome will thow an error – Hookworm 17/2, 2023 at 23:9

@Hookworm last time I checked it worked with relative paths – Torbernite 18/2, 2023 at 17:22

driver = webdriver.Chrome(chrome_options=chrome_options) should be replaced by: driver = webdriver.Chrome(options=chrome_options) Using selenium 4.15.2 – Blunge 14/12, 2023 at 15:35

Remember, you can only add a cookie for the current domain.

If you want to add a cookie for your Google account, do

browser.get('http://google.com')
for cookie in cookies:
    browser.add_cookie(cookie)

Detumescence answered 10/2, 2014 at 18:14 Comment(6)

This should be in their documentation :( – Obliquely 10/3, 2018 at 22:45

@Obliquely selenium-python.readthedocs.io/… – Jerrine 12/3, 2018 at 21:49

@MauricioCortazar it says nothing about the domain requirement, which is what I was referring to – Obliquely 13/3, 2018 at 3:4

@Obliquely that's basic man, the cookies only are stored in the domain, even subdomain aren't allowed – Jerrine 13/3, 2018 at 3:24

This comment seems relevant where it comes to multiple domains using a cookie from a root domain. For example, google.com could be the root domain, and another domain or subdomain owned by Google could use the same cookie. I like the solution by @Eduard Florinescu better because of this (and other reasons) as it doesn't require using the browser.get before loading cookies, they are just there already from the data dir. It seems the additional browser.get is required here before loading the cookies file (as per this comment), though did not test it. – Schauer 3/6, 2019 at 5:7

This just seems very backwards, if you set the cookies after you load the site then they won't be active when you load the site. How do you set cookies that will be active on the first call to browser.get? – Stricklin 11/7, 2022 at 20:46

Just a slight modification for the code written by Roel Van de Paar, as all credit goes to him. I am using this in Windows and it is working perfectly, both for setting and adding cookies:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--user-data-dir=chrome-data")
driver = webdriver.Chrome('chromedriver.exe',options=chrome_options)
driver.get('https://web.whatsapp.com')  # Already authenticated
time.sleep(30)

Halliard answered 10/8, 2019 at 18:5 Comment(1)

Worked for me, although I had to set the specific path (I use os.getcwd()) on user-data-dir. – Discord 26/9, 2021 at 19:29

Based on the answer by Eduard Florinescu, but with newer code and the missing imports added:

$ cat work-auth.py
#!/usr/bin/python3

# Setup:
# sudo apt-get install chromium-chromedriver
# sudo -H python3 -m pip install selenium

import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--user-data-dir=chrome-data")
driver = webdriver.Chrome('/usr/bin/chromedriver',options=chrome_options)
chrome_options.add_argument("user-data-dir=chrome-data")
driver.get('https://www.somedomainthatrequireslogin.com')
time.sleep(30)  # Time to enter credentials
driver.quit()

$ cat work.py
#!/usr/bin/python3

import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--user-data-dir=chrome-data")
driver = webdriver.Chrome('/usr/bin/chromedriver',options=chrome_options)
driver.get('https://www.somedomainthatrequireslogin.com')  # Already authenticated
time.sleep(10)
driver.quit()

Schauer answered 3/6, 2019 at 5:27 Comment(2)

The pickle stuff didn't work for me. (This is the second time I've tried using it.) So I used your method which also didn't work for me at first. Changes I had to make: I had to type chrome_options.add_argument('no-sandbox') due to the problem documented at github.com/theintern/intern/issues/878 and I had to make user-data-dir a full path in my Windows 10 environment. – Relentless 10/6, 2019 at 9:21

Not working for my website that stores authentication data in cookies – Enkindle 14/12, 2019 at 23:26

This is a solution that saves the profile directory for Firefox.

The advantage of this solution compared to the top answer using driver.get_cookies() is that, apart from cookies, other data (localStorage, IndexedDB) are also stored, which would be useful because a few websites use them to persist the session.

The solution using Chrome's user-data-dir also keep the localStorage similar to this, but it uses Chrome instead of Firefox.

It was tested on Linux.

Ideally it would be better to not copy the directory in the first place, but this is very hard, see

Also

Can't use existing Firefox profile in Selenium WebDriver using C# (similar solution to the solution below)

Short version:

To save the profile

driver.execute_script("window.close()")
time.sleep(0.5)
currentProfilePath = driver.capabilities["moz:profile"]
profileStoragePath = "/tmp/abc"
shutil.copytree(currentProfilePath, profileStoragePath,
                ignore_dangling_symlinks=True
                )

To load the profile

driver = Firefox(executable_path="geckodriver-v0.28.0-linux64",
                 firefox_profile=FirefoxProfile(profileStoragePath)
                )

Long version (with demonstration that it works and a lot of explanation -- see comments in the code)

The code uses localStorage for demonstration, but it works with cookies as well.

#initial imports

from selenium.webdriver import Firefox, FirefoxProfile

import shutil
import os.path
import time

# Create a new profile

driver = Firefox(executable_path="geckodriver-v0.28.0-linux64",
                  # * I'm using this particular version. If yours is
                  # named "geckodriver" and placed in system PATH
                  # then this is not necessary
                )

# Navigate to an arbitrary page and set some local storage
driver.get("https://DuckDuckGo.com")
assert driver.execute_script(r"""{
        const tmp = localStorage.a; localStorage.a="1";
        return [tmp, localStorage.a]
    }""") == [None, "1"]

# Make sure that the browser writes the data to profile directory.
# Choose one of the below methods
if 0:
    # Wait for some time for Firefox to flush the local storage to disk.
    # It's a long time. I tried 3 seconds and it doesn't work.
    time.sleep(10)

elif 1:
    # Alternatively:
    driver.execute_script("window.close()")
    # NOTE: It might not work if there are multiple windows!

    # Wait for a bit for the browser to clean up
    # (shutil.copytree might throw some weird error if the source directory changes while copying)
    time.sleep(0.5)

else:
    pass
    # I haven't been able to find any other, more elegant way.
    #`close()` and `quit()` both delete the profile directory


# Copy the profile directory (must be done BEFORE driver.quit()!)
currentProfilePath = driver.capabilities["moz:profile"]
assert os.path.isdir(currentProfilePath)
profileStoragePath = "/tmp/abc"
try:
    shutil.rmtree(profileStoragePath)
except FileNotFoundError:
    pass

shutil.copytree(currentProfilePath, profileStoragePath,
                ignore_dangling_symlinks=True # There's a lock file in the
                                              # profile directory that symlinks
                                              # to some IP address + port
               )

driver.quit()
assert not os.path.isdir(currentProfilePath)
# Selenium cleans up properly if driver.quit() is called,
# but not necessarily if the object is destructed


# Now reopen it with the old profile

driver=Firefox(executable_path="geckodriver-v0.28.0-linux64",
               firefox_profile=FirefoxProfile(profileStoragePath)
              )

# Note that the profile directory is **copied** -- see FirefoxProfile documentation
assert driver.profile.path!=profileStoragePath
assert driver.capabilities["moz:profile"]!=profileStoragePath

# Confusingly...
assert driver.profile.path!=driver.capabilities["moz:profile"]
# And only the latter is updated.
# To save it again, use the same method as previously mentioned

# Check the data is still there

driver.get("https://DuckDuckGo.com")

data = driver.execute_script(r"""return localStorage.a""")
assert data=="1", data

driver.quit()

assert not os.path.isdir(driver.capabilities["moz:profile"])
assert not os.path.isdir(driver.profile.path)

What doesn't work:

Initialize Firefox(capabilities={"moz:profile": "/path/to/directory"}) -- the driver will not be able to connect.
options=Options(); options.add_argument("profile"); options.add_argument("/path/to/directory"); Firefox(options=options) -- same as above.

Audiology answered 2/1, 2021 at 3:36 Comment(1)

Alternatively, you can persist only the cookies.sqlite database (or equivalent) rather than the entire profile. See this answer. – Slogan 11/11, 2023 at 22:13

This is code I used in Windows. It works.

for item in COOKIES.split(';'):
    name,value = item.split('=', 1)
    name=name.replace(' ', '').replace('\r', '').replace('\n', '')
    value = value.replace(' ', '').replace('\r', '').replace('\n', '')
    cookie_dict={
            'name':name,
            'value':value,
            "domain": "",  # Google Chrome
            "expires": "",
            'path': '/',
            'httpOnly': False,
            'HostOnly': False,
            'Secure': False
        }
    self.driver_.add_cookie(cookie_dict)

Augsburg answered 15/8, 2019 at 9:43 Comment(0)

Try this method:

import pickle
from selenium import webdriver
driver = webdriver.Chrome(executable_path="chromedriver.exe")
URL = "SITE URL"
driver.get(URL)
sleep(10)
if os.path.exists('cookies.pkl'):
    cookies = pickle.load(open("cookies.pkl", "rb"))
    for cookie in cookies:
        driver.add_cookie(cookie)
    driver.refresh()
    sleep(5)
# check if still need login
# if yes:
# write login code
# when login success save cookies using
pickle.dump(driver.get_cookies(), open("cookies.pkl", "wb"))

Decapolis answered 14/6, 2021 at 15:10 Comment(0)

Use this code to store the login session of any website like google facebook etc

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import undetected_chromedriver as uc
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:/Users/salee/AppData/Local/Google/Chrome/User Data/Profile 1")
browser = uc.Chrome(use_subprocess=True,Options=options)

Sumter answered 27/10, 2022 at 14:45 Comment(0)

For my case, the accepted answer is almost there.

For people that have no luck with the answers above, you are welcome to try my way.

Before you start coding, make sure that the website is using cookies for authentication.

Step:

Open your browser (I am using chrome here), login to your website.
Go to this website in order to know how to check the value of the cookies
Open another browser in incognito mode and go to your website (at this stage, your website should still prompt you the login page)
Try to modify the cookies value with the first browser cookies's value accordingly (your first browser must authenticated to your website)
Refresh your incognito mode browser, it should by pass the login page

The steps above is how I used to make sure adding cookies can authenticate to my website.

Now is the coding part, it is almost the same as the accepted answer. The only problem for me with the accepted answer is that I ended up having double the numbers of my cookies.

The pickle.dump part has no issue for me, so I would straight to the add cookie part.

import pickle
import selenium.webdriver

driver = selenium.webdriver.Chrome()
driver.get("http://your.website.com")

cookies = pickle.load(open("cookies.pkl", "rb"))
# the reason that I delete the cookies is because I found duplicated cookies by inspect the cookies with browser like step 2
driver.delete_all_cookies()

for cookie in cookies:
    driver.add_cookie(cookie)

driver.refresh()

You are able to use step 2 to check if the cookies you add with the code is working correctly.

Hope it helps.

Lethia answered 22/12, 2022 at 7:31 Comment(0)

Most of the answers here are about pickling the cookies, I tried it and it didn't work properly. you can simply store the cookies() response in a text file and load it when you need it.

Go to the website and login or do other cookie related activities

from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://google.com")

Store the cookie

with open("cookies.txt", "r") as f:
    cookies = eval(f.read())

to test it, close and reopen the driver and load the cookie

driver.quit()
driver = webdriver.Chrome()
driver.get("https://google.com")

Load the cookie

for cookie in cookies:
    driver.add_cookie(cookie)

refresh to reflect the change

driver.refresh()

Cleodel answered 8/4, 2023 at 20:41 Comment(0)

None of the answers here solved my problem so I figured that I would share the method that I discovered that ended up being my solution.

Of course, you need to first save your session cookies:

def get_cookies(self):
    cookies = {}
    selenium_cookies = self.driver.get_cookies()
    for cookie in selenium_cookies:
        cookies[cookie['name']] = cookie['value']
    return cookies
def dump_cookies(self, output_file_path: str):
    cookies = self.get_cookies()
    with open(output_file_path, 'w') as f:
        json.dump(cookies, f)
    self.debug_print(f'Saved cookies to: {output_file_path}')

It is usually enough to simply load those cookies and then refresh the page:

def load_cookies(self, path):
    with open(path) as f:
        cookies = json.load(f)
    for key, value in cookies.items():
        self.driver.add_cookie({'name': key, 'value': value})

But in my case, I also had to save the local storage ... :

def save_storage(self):
    local_storage = self.driver.execute_script(
        "var items = {}, ls = window.localStorage; for (var i = 0; i < ls.length; i++)  items[ls.key(i)] = ls.getItem(ls.key(i)); return items;")
    with open("data/local_storage.json", "w") as f:
        json.dump(local_storage, f)

And then to load that storage in your fresh session:

def load_storage(self):
    with open("data/local_storage.json", "r") as f:
        local_storage = json.load(f)
    for key, value in local_storage.items():
        self.driver.execute_script(f"window.localStorage.setItem('{key}', '{value}')")

Quick recap:

Load selenium, navigate to your site that requires auth.
Run dump_cookies and save_storage to save the session data to disc.
Close the current session
Load a new session, navigate to that same site again.
Run load_cookies and load_storage and then driver.refresh() to refresh the page.
Hopefully you will now be logged in. If not, try running driver.delete_all_cookies() before loading the cookies, as this will clear any cookies that the site may have set before you loaded the login cookies. This was not necessary for me.

Hobbism answered 1/9, 2023 at 7:41 Comment(0)

For OS = Mac OS Ventura; Python = 3.10.4; SE = 4.12.0, check out this one that I was able to test very recently Connecting to an Edge Session and profile

Avidin answered 21/9, 2023 at 16:49 Comment(1)

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review – Geibel 26/9, 2023 at 17:46

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Recommended topics

Hot tags