How to pass environment variables to pytest

Question

Before I start executing the tests in my Python project, I read some environment variables and set some variables with these values read. My tests will run on the desired environment based on these values read.

E.g., Let's say the environment variables are called ENV_NAME and ENV_NUMBER. I would like to run the tests using pytest.

If I hard code these environment variables (e.g.,ENV_NAME = 'staging', ENV_NUMBER = '5') in my code and then run the tests by executing the pytest command at the root of the project directory, all the tests run successfully.

But I don't want to hardcode these values. Is there a way I can set these environment variables as command line arguments for pytest?

I was thinking more in the lines of

pytest -ENV_NAME='staging' -ENV_NUMBER='5'.

But this is not working.

Rema

This is a similar question that may help: #54901285 – Fratricide

Answer 1

I finally found the answer i was looking for.

we can set the environment variables like this before running tests using py.test

ENV_NAME='staging' ENV_NUMBER='5' py.test

Rema

What a solution, to set environment variables, set environment variables. Was hoping for a real solution … – Ciel

Answer 2

Another alternative is to use the pytest-env plugin. It can be configured like so:

[pytest]
env = 
    HOME=~/tmp
    D:RUN_ENV=test

the D: prefix allows setting a default value, and not override existing variables passed to py.test.

Note: you can explicitly run pytest with a custom config, if you only sometimes need to run a specialized environment set up:

pytest -c custom_pytest.ini

---

If you use PyCharm vs pytest-dotenv, this may be helpful

Northwest

Using pytest-dotenv successfully took the environment variables defined in my .env file and made them available when running pytest. – Knickerbockers

pytest-env looks defunct, while python-dotenv is still active. – Aeriel

@Aeriel are you experiencing any issues using pytest-env, or just noticed it's not been updated for a while (wow for 5 years)? python-dotenv has a slightly different set of use cases; I'd love to hear how you use to update env vars with pytest 🙏 – Northwest

@Northwest It was just an alert to the newcomer (like me), since this answer is so highly voted. I've never used pytest-env (but I've considered for a while). Right now I'm setting my env vars as prescribed by python-dotenv. I have a .env file in the root directory, and in the test modules that need to use those variables I call load_dotenv(). – Aeriel

It looks like pytest-env was updated to add type hints in Oct 2022. – Prana

Answer 3

In addition to other answers. There is an option to overwrite pytest_generate_tests in conftest.py and set ENV variables there.

For example, add following into conftest.py:

import os

def pytest_generate_tests(metafunc):
    os.environ['TEST_NAME'] = 'My super test name| Python version {}'.format(python_version)

This code will allow you to grab TEST_NAME ENV variable in your tests application. Also you could make a fixture:

import os
import pytest

@pytest.fixture
def the_name():
    return os.environ.get('TEST_NAME')

Also, this ENV variable will be available in your application.

Open

In conftest.py, you don't even have to use pytest_generate_tests; you can simply set os.environ. – Fdic

Answer 4

1. I use monkey patch when I don't load environment variable variable outside function.

import os

# success.py
def hello_world():
    return os.environ["HELLO"]

# fail.py
global_ref = os.environ["HELLO"] # KeyError occurs this line because getting environment variable before monkeypatching

def hello_world():
    return global_ref

# test.py
def test_hello_world(monkeypatch):
    # Setup
    envs = {
        'HELLO': 'world'
    }
    monkeypatch.setattr(os, 'environ', envs)

    # Test
    result = hello_world()

    # Verify
    assert(result == 'world')

2. If you use PyCharm you can set environment variables, [Run] -> [Edit Configuration] -> [Defaults] -> [py.tests] -> [Environment Variables]

Sanguinary

For another example of monkeypatch, see: aalvarez.me/posts/pytest-tricks-for-better-python-tests – Dispersive

Answer 5

It is possible to use autouse fixtures.

@pytest.fixture(scope="session", autouse=True)
def set_env():
    os.environ["FLAG"] = "1"

Obnoxious

Awesome this is definitely the most elegant and idiomatic solution! – Congress

I was adding environment variables by simply having os.environ['FLAG'] = '1' at the top of my conftest.py, outside of any function. I am curious - what is the benefit of adding it in a pytest fixture function? – Nitrogenize

@crabulus_maximus, 1. Scoped Setup: Fixtures can limit environment changes to specific scopes, preventing side effects across tests. 2. Delayed Execution: The environment setup isn't run until the fixture is invoked, useful for dynamic values or selective test runs. 3. Teardown Logic: Fixtures allow for automatic cleanup, unsetting variables, or restoring states, ensuring isolation between tests. 4. Readability: It signals to others that certain tests depend on these environment setups, improving clarity. This method enhances control and reliability, especially in extensive test suites. – Obnoxious

I've spent a while figuring out why this didn't reliably work for me, and it was because this changes the actual environment values. The better way to write the fixture is with monkeypatch: docs.pytest.org/en/6.2.x/… – Iced

Answer 6

There are few ways you can achieve this

1. If you dont want to use the environment variable , you can use pytest addoptions as https://docs.pytest.org/en/latest/example/simple.html

2. You can write a wrapper script like this to call enviornment variables

   import os
   import py
   env_name = os.environ["ENV_NAME"]
   env_no = os.environ["ENV_NUMBER"]
   pytest_args=(env_name,env_no)
   pytest.main('-s' ,pytest_args,test_file.py) 
   

in test_file.py you can use

   env_n, env_n = pytest.config.getoption('pytest_args')

  

3. Alternate method if you just want to pass the date not set enviornment variable

on command line you can use it as

   py.test --testdata ="ENV_NAME:staging,ENV_NUMBER:5"

You can use in your test file

pytest_params = pytest.config.getoption('testdata')
params = pytest_params.split(":")
param_dict = dict(params[i:i+2] for i in range(0,len(params),2))
env_name = param_dict["ENV_Name"]

Peasecod

Looks like the 1. is save as 3. – Cuda

Almost !! Except the first one is fixture called from conftest second one is called directly from test. – Peasecod

@Peasecod could you give more info on technique 2 please? Would this wrapper be able to conditionally assign env vars to specific tests in a directory? – Stutter

@Stutter , do you mean specific test file or tests within a file ? whatever args you pass in pytest.main('-s' ,pytest_args,test_file.py), should be available in test_file.py. If that's not clear could you elaborate on your requirement ? – Peasecod

Answer 7

Following the idea provided by @tutuDajuju using pytest-env - an alternative would be to write a custom plugin leveraging pytest_load_initial_conftests. Might be useful especially when you don't want or can't install external dependencies.

Here's a quick example:

Project structure

.
├── __init__.py
├── pytest.ini
├── script.py
└── tests
    ├── __init__.py
    ├── plugins
    │   ├── __init__.py
    │   └── env_vars.py
    └── test_script.py

script.py

import os

FOOBAR = os.environ.get("FOOBAR")


def foobar():
    return FOOBAR

test_script.py

from script import foobar


def test_foobar():
    assert foobar() == "foobar"

pytest.ini

[pytest]
addopts = -p tests.plugins.env_vars

env_vars.py

import os

import pytest


@pytest.hookimpl(tryfirst=True)
def pytest_load_initial_conftests(args, early_config, parser):
    os.environ["FOOBAR"] = "foobar"

Example run:

$ python -m pytest tests -v
========= test session starts =========
platform darwin -- Python 3.8.1, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- 
rootdir: /Users/user/pytest_plugins, inifile: pytest.ini
collected 1 item

tests/test_script.py::test_foobar PASSED                                                                                               [100%]

========= 1 passed in 0.01s =========

Listing

Answer 8

Run export inside a subshell (enclosing parenthesis) not to mess up local environment. Supply export with parameters from .env file.

(export $(xargs < .env); pytest -svvvx api)

Gnatcatcher

Answer 9

Similar to bad_coder had mentioned, you can do:

# test.py
def test_hello_world(monkeypatch):
    # Setup
    monkeypatch.setenv('HELLO', 'world')

    # Test
    result = hello_world()

    # Verify
    assert(result == 'world')

Manganite

Totally seems like the right answer but, just didn't work for me, and I can't work out why. – Trembly

@JohnMee If you need the environment variables at import time, that might be the reason why monkeypatching fails – Wallford

This is the way of doing it, and pytest documentation takes a tiny bit further creating fixtures out of this. docs.pytest.org/en/6.2.x/… – Iced

Answer 10

I needed to create a pytest.ini file and pass the environment variables to the pytest command. E.g:

In the pytest.ini file I set an empty value because it is overwritten by whatever you pass to the command line command:

[pytest]
MY_ENV_VAR=

Command line, with the actual value set:

$ MY_ENV_VAR=something pytest -c pytest.ini -s tests/**

I don't know why does it work like this. I just found out that it works as a result of mere trial and error, because the other answers didn't help me.

Perkoff

@Acumenus As you say, it seems. But the actuality is other, since removing the entry in pytest.ini will cause the CLI env var to be ignored. – Perkoff

The environment variable as it is defined on the CLI will be available to the entire Python process, and nothing about it is pytest specific. For the sake of argument, consider $ FOO=bar python -c 'import os; print(os.environ["FOO"])'. If the CLI env var is being ignored, then your application must not be accessing it as an env var, but in some other way. – Racial

@Acumenus My answer is pytest specific. What you say is all correct in theory but, as I explained in my answer, I don't know why pytest will need that workaround, regarding the env vars, but it does need that workaround. – Perkoff

I had the same issue as you. I created a simple shell script run-test.sh in my test directory. It contains the following one-liner: ENV_VAR_NAME_1="env_var_value_1" ENV_VAR_NAME_2="env_var_value_2" pytest -c path/to/pytest.ini path/to/test/ It's essential to have it on one line; otherwise, the environment variable won't be loaded by pytest. Enable the shell script execution with chmod +x run-test.sh. You can now run your pytest tests by running ./run-test.sh. – Dang

Answer 11

Although the other answer works I think this one is more "hands-off" and automated and it simulates normal operation more. So I use python-dotenv to load all variables form a file with load_dotenv(my_filepath):

import os
import pytest
from dotenv import load_dotenv
from core import ConfigService


def test_config_service():
    """This test ensures that the config service can read the environment
    variables which define the location of the config.json files"""

    load_dotenv("env/common.env")
    config_service = ConfigService()
    config_service.load()
    assert config_service.config_folder_path is not None
    assert config_service.config_folder_name is not None

I think it is better if you want to test your whole logic of:

• Reading the variable from an .env file in a specific location and
• Checking to see if the code you are testing is performing as expected based on the values in that file (maybe you could catch typos or other problems with your logic)

Cheadle

I believe this is only good if you have static variables that load at the time of execution. If, however, you want to pass and interact with variables passed from terminal / command line arguments then I believe you'd need an approach like: def pytest_addoption(parser): parser.addoption('--remote', action='store', default='False', help='run ui tests remotely') in a conftest.py file. – Powys

Environment variables for testing must be static. Make a test for each permutation, otherwise it ain't testing with proper coverage. – Medici

@CanH.Tartanoglu I have a .env file with common environment variables used in all environments, this way I test if they are read in correctly and also validate their values. I dont understand what you mean they have to be static. Also there are no permutations. I miss something here? – Cheadle

You can't use your own environment variables for testing and call it testing with full coverage. You need to test with EVERY acceptable value for ALL environment variables used by your application (there are some exceptions to this rule). Assuming you want testing with full coverage, which every developer should strive for. Meaning I wouldn't use the dotenv method either, but your critique was alarming to me. – Medici

But no. I set the environment variables. The values in my .env are what their values are. If they ever change they will change from that very file that is the whole point no? Where would these wild environment variable values will come from? Centrally managing and testing environment variable is the whole point. – Cheadle

Obviously I am not refering to constant environment variables such as file or directory paths, but logical environment variables such as booleans and integers. But why have a .env file just for testing when you could just define the variables inside pytest? – Medici

@CanH.Tartanoglu but that is what I describe here. I named the file common cause they are common in all environments and if they change I want them to change from that very file, so I have control over them. So different cases really. – Cheadle

It is an unjustified additional layer of complexity. – Medici

@CanH.Tartanoglu that is a very personal judgement without taking into account the purpose of what we are doing there. But each one to their own. – Cheadle

Sad to hear that, I tried to be constructive without emotional judgement. Please read my comments again, cheers. – Medici

Answer 12

I'm using pytest-dotenv plugin to load .env file and it works pretty well. You can mention multiple env files in your pytest.ini like this.

[pytest]
env_files =
    .env
    .test.env
    .deploy.env

Leon

Answer 13

If you are using some conftest.py files and need to define different values to one environment variable, you can use the monkeypatch like this:

# tests/conftest.py
import pytest
@pytest.fixture(autouse=True)
def env_setup(monkeypatch):
    monkeypatch.setenv("ENVIRONMENT", "local")

The autouse=True allow you use this fixture in your tests that localized in the same conftest's folder without add manually.

Example:

# tests/my_test.py
import os

def is_local_env():
    if os.getenv("ENVIRONMENT") == "local":
        return True
    return False

def test_if_is_local_env():
    result = is_local_env()
    assert result is True

Psychedelic

Answer 14

You can just patch environment variables os.environ using patch.dict.

For your sepecific case, to patch env vars before tests execution, you can place the following code in conftest.py:

from unittest.mock import patch
patch.dict(os.environ, {"ENV_NAME": "staging", "ENV_NUMBER": "5"}).start()

Orme

Answer 15

You could use monkeypatching, which is bundled with pytest when your environment variables are dynamic.

Medici