Set environment variable of calling bash script in python [duplicate]

Question

I have a bash script that looks like this:

python myPythonScript.py

python myOtherScript.py $VarFromFirstScript

and myPythonScript.py looks like this:

print("Running some code...")
VarFromFirstScript = someFunc()
print("Now I do other stuff")

The question is, how do I get the variable VarFromFirstScript back to the bash script that called myPythonScript.py.

I tried os.environ['VarFromFirstScript'] = VarFromFirstScript but this doesn't work (I assume this means that the python environment is a different env from the calling bash script).

Monday

You can't change environment variables of other processes. You can only influence them when starting a process, so environment variables are not an option. That leaves various other ways of IPC, but the most natural one is probably to write to stdout. – Selfimmolation

so given that the python script has a bunch of print statements i'd just have to parse all of them in bash to get the one export statement i want? – Monday

Why does it have all those print() calls (not statements, not in Python 3!)? I guess it's for logging, right? In that case, firstly, use a logging library, which you could then use to log to a file, too. Secondly, don't use stdout for that but stderr, which is generally used for non-payload, informal output. – Selfimmolation

Answer 1

you cannot propagate an environment variable to the parent process. But you can print the variable, and assign it back to the variable name from your shell:

VarFromFirstScript=$(python myOtherScript.py $VarFromFirstScript)

you must not print anything else in your code, or using stderr

sys.stderr.write("Running some code...\n")
VarFromFirstScript = someFunc()
sys.stdout.write(VarFromFirstScript)

an alternative would be to create a file with the variables to set, and make it parse by your shell (you could create a shell that the parent shell would source)

import shlex
with open("shell_to_source.sh","w") as f:
   f.write("VarFromFirstScript={}\n".format(shlex.quote(VarFromFirstScript))

(shlex.quote allows to avoid code injection from python, courtesy Charles Duffy)

then after calling python:

source ./shell_to_source.sh

Liaoning

This should be pipes.quote(VarFromFirstScript) (or, in Python 3, shlex.quote(...)) to avoid security issues. – Robillard

It's also common to write to stdout -- consider how ssh-agent is expected to have its output invoked w/ eval "$(ssh-agent -s)". The suggestion to write to a file and source the file seems rather needless here. – Robillard

ok but this method allows to change several variables. – Bioclimatology

I'm not sure I understand what you're saying there. Both the eval method and the source method are identical in that respect; ssh-agent definitely sets more than one variable. Keep in mind that you can eval a string with code that contains literal newlines. – Robillard

Answer 2

You can only pass environment variables from parent process to child.

When the child process is created the environment block is copied to the child - the child has a copy, so any changes in the child process only affects the child's copy (and any further children which it creates).

To communicate with the parent the simplest way is to use command substitution in bash where we capture stdout:

Bash script:

#!/bin/bash
var=$(python myPythonScript.py)
echo "Value in bash: $var"

Python script:

print("Hollow world!")

Sample run:

$ bash gash.sh
Value in bash: Hollow world!

You have other print statements in python, you will need to filter out to only the data you require, possibly by marking the data with a well-known prefix.

If you have many print statements in python then this solution is not scalable, so you might need to use process substitution, like this:

Bash script:

#!/bin/bash

while read -r line
do
    if [[ $line = ++++* ]]
    then
        # Strip out the marker
        var=${line#++++}
    else
        echo "$line"
    fi
done < <(python myPythonScript.py)

echo "Value in bash: $var"

Python script:

def someFunc():
    return "Hollow World"

print("Running some code...")

VarFromFirstScript = someFunc()
# Prefix our data with a well-known marker
print("++++" + VarFromFirstScript)

print("Now I do other stuff")

Sample Run:

$ bash gash.sh
Running some code...
Now I do other stuff
Value in bash: Hollow World

Diannadianne

Answer 3

I would source your script, this is the most commonly used method. This executes the script under the current shell instead of loading another one. Because this uses same shell env variables you set will be accessible when it exits. . /path/to/script.sh or source /path/to/script.sh will both work, . works where source doesn't sometimes.

Jaclyn

If that code only was a shell script. You can't source Python code though. – Selfimmolation

True, use subprocess to create bash env variable from python script then? – Jaclyn

As I understand it, BASH is already running as parent process and there's no way to change another process' environment variables. – Selfimmolation