Can't find dotenv environment variables from cron job

Question

I am running a python script from a python library which loads some environment variables from a .env file in the root of the library using dotenv.

This works from the command line, but when I try to run as a cronjob using the following:

* * * * * source ./path_to_venv/activate; python ./path_to_script.py

I get a key error because it can't find the environment variable.

Any ideas why this isn't working?

Many thanks for any help!

Modal

Answer 1

I'm using crontab as well to execute my Node JS project. I have to explicitly state the path of my .env file like so:

require('dotenv').config({ path: '/var/www/html/myproject/.env' });

In python-dotenv, I believe it can be done similarly by using:

# OR, explicitly providing path to '.env'
from pathlib import Path  # Python 3.6+ only
env_path = Path('.') / '.env'
load_dotenv(dotenv_path=env_path)

Source

Deeplaid

Answer 2

If you're using dotenvs default config path, it is resolved from path.resolve(process.cwd(), '.env'), which when running your script from cron, will not resolve to what you expect.

So, use this example if your .env file is in the same level as your script:

const dotenv = require('dotenv')
dotenv.config({ path: __dirname + '/.env' })

Ridings

Answer 3

I don't know if there's a more elegant solution with this one. I was able to make my script work in crontab by adding the environment variables at the top of the cronjob.

API_KEY=value
API_KEY_SECRET=value
ACCESS_TOKEN=value=value
ACCESS_TOKEN_SECRET=value

# run on 8hrs interval
0 */8 * * * . $HOME/Coding/python/web-scraper-corona/venv/bin/activate && $HOME/Coding/python/web-scraper-corona/venv/bin/python3 /home/chan-dev/Coding/python/web-scraper-corona/twitter-covid19-bot.py >> /tmp/test.txt 2>&1

Pinson

Answer 4

Node.js ES6-way without any external modules:

// env.js
import dotenv from 'dotenv'
const _dirname = new URL('.', import.meta.url).pathname
dotenv.config({ path: _dirname + '.env' })

// yourmodule.js
import _ from './env.js'

// use process.env

Glutamate

Answer 5

Several issues if you're struggling with several factors like I was. True, pure Linux sysadmin, it's a hassle to load env from cron and probably a security risk, even in a container. But if you're using dotenv, the issue is probably a combination of loading the absolute path in dotenv (crontab doesn't know where your .env is), some basic logging (to see what the real error is) and finally the crontab user needs pip installed!

Try:

1. Explicitly set the path to env in the python script: envpath = join(dirname('/var/www/node-folder/approot/'), '.env')
2. In crontab, make sure you log the errors! * * * * * sudo -H PYTHONPATH=/usr/bin/python3 /usr/bin/python3 /var/www/node-folder/approot/app.py >> /var/log/app.log 2>&1
3. Guess what? sudo or whatever user might not have the pips!

Example:

    from sqlalchemy import create_engine,text
ModuleNotFoundError: No module named 'sqlalchemy'

Cervix