How to set virtualenv for a crontab?

Question

I want to set up a crontab to run a Python script.

Say the script is something like:

#!/usr/bin/python
print "hello world"

Is there a way I could specify a virtualenv for that Python script to run in? In shell I'd just do:

~$ workon myenv

Is there something equivalent I could do in crontab to activate a virtualenv?

Frons

Answer 1

If you're using "workon" you're actually using "virtualenv wrapper" which is another layer of abstraction that sits on top of virtualenv. virtualenv alone can be activated by cd'ing to your virtualenv root directory and running:

source bin/activate

workon is a command provided by virtualenv wrapper, not virtualenv, and it does some additional stuff that is not necessarily required for plain virtualenv. All you really need to do is source the bin/activate file in your virtualenv root directory to "activate" a virtualenv.

You can setup your crontab to invoke a bash script which does this:

#! /bin/bash    
cd my/virtual/env/root/dir
source bin/activate

# virtualenv is now active, which means your PATH has been modified.
# Don't try to run python from /usr/bin/python, just run "python" and
# let the PATH figure out which version to run (based on what your
# virtualenv has configured).

python myScript.py

Aircrew

Do I still need to use "#!/usr/bin/python" to specify my Python interpreter in my script? But my virtualenv might point to a different interpreter. This is where I'm confused. – Frons

You may want to take a look at what bin/activate is doing. Activating a virtualenv is basically just modifying your PATH env var to point to specific versions of commands, like python, etc. If you activate a virtualenv, then try to run /usr/bin/python, you may or may not be using the version of python that your virtualenv is expecting. Rather than doing "#!/usr/bin/python" you can do "#!/usr/bin/env python" to let the env decide which python to run, based on your PATH. – Aircrew

Just as a heads up, don't try source in your cron line, as its a bashism and won't work since cron will use /bin/sh to execute your command. – Incapacitate

make sure that the #! /bin/bash is the very first line in the file as well, no whitespace before it – Gazelle

It's not true to that all workon does is source the bin/activate file. Virtualenvwrapper also has hooks you can use (e.g. postactivate) which won't be run if you just source the activate file. – Tryptophan

Go with this script if using any custom email templates,custom paths, because checking would be very easy and it's scaleable – Karmen

"don't try source in your cron line, as its a bashism and won't work since cron will use /bin/sh to execute your command." You can use . command in sh though. See #4151171 – Querist

Answer 2

Is there something equivalent I could do in crontab to activate a virtualenv?

This works well for me...

## call virtualenv python from crontab
0    9    *    *    *    /path/to/virtenv/bin/python /path/to/your_cron_script.py

I prefer using python directly from the virtualenv instead of hard-coding the virtualenv $PATH into the script's shebang... or sourcing the venv activate

Thalia

This works really well. In fact, it is also possible to put the absolute path to the python interpreter in the shebang (aka #!) of the script itself and, of course, make the script executable. – Tunny

@AvinashMeetoo Yes, it's possible but this way it becomes location- and setup-dependan – Collyer

Tried this, and found that my script couldn't import from other modules in the same project. Any ideas why that might happen? – Presage

@NathanGould you probably needed to add a .pth file in your virtual environment virtual_env_folder/lib/python3.3/site-packages/app_name.pth with the location of your app as directed here https://mcmap.net/q/169056/-how-do-i-add-a-path-to-pythonpath-in-virtualenv – Allamerican

pyvenv-3.3 under Ubuntu 13.10 and Debian is buggy. See bug 1093193 and bug 1298831, and see the reason and workaround (just add a symlink venvdir/lib/python3.3/dist-packages pointing to site-packages) – Faubourg

This is the better solution – Whiteman

What the heck is 0 9 * * * Can you explain that notation, i came to this post from anon crontab post. – Mcqueen

@user1767754, see man 5 crontab – Thalia

Answer 3

With bash, you can create a generic virtual env wrapper that you can use to invoke any command, much like how time can wrapper any command.

virt_env_wrapper.bash:

#!/bin/bash    
source path/to/virtual/env/bin/activate
"$@"

Bash's magical incantation "$@" re-escapes all tokens on the original command line so that if you were to invoke:

virt_env_wrapper.bash python foo.py bar 'baz blap'

foo.py would see a sys.argv of ['bar', 'baz blap']

Semifinal

Answer 4

I'm not sure about workon, but it's pretty straightforward for venv. The only thing to remember is that crontab uses sh by default, not bash, so you need to use the . command instead of source.

Here are examples if you have a file ~/myproject/main.py:

* * * * * cd ~/myproject && . .venv/bin/activate && python main.py > /tmp/out1 2>&1

You could also directly call the specific path of the python in the venv directory, then you don't need to call activate.

* * * * * ~/myproject/.venv/bin/python ~/myproject/main.py > /tmp/out2 2>&1

The downside of that is you would need to specify the project path twice, which makes maintenance trickier. To avoid that, you could use a shell variable so you only specify the project path once:

* * * * * project_dir=~/myproject ; $project_dir/.venv/bin/python $project_dir/main.py > /tmp/out3 2>&1

Querist