Why do people write "#!/usr/bin/env python" on the first line of a Python script?

Question

I see these at the top of Python files:

#!/usr/bin/env python

#!/usr/bin/env python3

It seems to me that the files run the same without that line.

Tactful

possible duplicate of What's the difference between these two python shebangs – Kos

I'm voting to close as a duplicate since none of the top voted answers here actually answer the question why there is a shebang. Does this answer your question? Should I put #! (shebang) in Python scripts, and what form should it take? – Singlecross

Answer 1

If you have several versions of Python installed, /usr/bin/env will ensure the interpreter used is the first one on your environment's $PATH. The alternative would be to hard code something like #!/usr/bin/python; that's ok, but less flexible.

In Unix, an executable file that's meant to be interpreted can indicate what interpreter to use by having a #! at the start of the first line, followed by the interpreter (and any flags it may need).

If you're talking about other platforms, of course, this rule does not apply (but that "shebang line" does no harm, and will help if you ever copy that script to a platform with a Unix base, such as Linux, Mac, etc.).

Bactria

Just to add: this applies when you run it in Unix by making it executable (chmod +x myscript.py) and then running it directly: ./myscript.py, rather than just python myscript.py. – Transmigrant

using env gives maximum flexibility in that the user can select the interpreter to use by changing the PATH. Often this flexibility is not required though and the downside is that linux for example can't use the script name for the name of the process in ps and reverts to "python". When packaging python apps for distros for example I would advise not to use env. – Subjoin

py launcher can use the shebang line on Windows. It is included in Python 3.3 or it can be installed independently. – Arabele

Note that you have to have a full path to an interpreter in the shebang. If not #!python would have sufficed instead of #!/usr/bin/env python – Stairwell

@Aneesh: This is true for UNIX, but not for Windows. Read the python.org/dev/peps/pep-0397 – Odoriferous

@Craig McQueen: I observe that calling python script ./myscript.py works but is different from python myscript.py since the former may have problems with module imports. – Pegpega

@Alex, All these added complexity is really pointless. Isn't it easier to create a symlink for /usr/bin/python? – Calumniation

@pixelbeat, Isn't smylink also equally flexible? – Calumniation

Generally one doesn't have permissions to edit a symlink, while they can control their $PATH – Subjoin

An important word of warning, env's return value eventually expires. Which is unlikely to affect you if you are running short-lived processes. However, I've had processes dying with the message /usr/bin/env: Key has expired after many hours. – Oviduct

There is a difference in location of env itself. For e.g on Ubuntu the location in /usr/bin/env whereas in CentOS it is /bin/env. What to do about that? – Untoward

@rsjethani: Every CentOS system I've used has env at /usr/bin/env (it may be at /bin/env, but it's also at /usr/bin/env). I've largely been on CentOS 6 though, did CentOS 7 change this behavior? – Vikki

If you invoke your script as python ./myscript.py, then the shebang line is ignored. Shell launches the python interpreter based on $PATH and passes the script path as the argument. Similarly, if we have a shell script invoked as source ./myscript.sh or sh ./myscript.sh or . ./myscript.sh, then the shebang line is inconsequential. – Lug

@Oviduct can you link to any resources that explain this expiry behavior? I can't find them. – Charlesettacharleston

Answer 2

That is called the shebang line. As the Wikipedia entry explains:

In computing, a shebang (also called a hashbang, hashpling, pound bang, or crunchbang) refers to the characters "#!" when they are the first two characters in an interpreter directive as the first line of a text file. In a Unix-like operating system, the program loader takes the presence of these two characters as an indication that the file is a script, and tries to execute that script using the interpreter specified by the rest of the first line in the file.

See also the Unix FAQ entry.

Even on Windows, where the shebang line does not determine the interpreter to be run, you can pass options to the interpreter by specifying them on the shebang line. I find it useful to keep a generic shebang line in one-off scripts (such as the ones I write when answering questions on Stack Overflow), so I can quickly test them on both Windows and Arch Linux.

The env utility allows you to invoke a command on the path:

The first remaining argument specifies the program name to invoke; it is searched for according to the PATH environment variable. Any remaining arguments are passed as arguments to that program.

Asseveration

@Arafangion you'll probably find this question useful. TL;DR: symbolhound.com – Sibling

"Even on Windows, where the shebang line does not determine the interpreter to be run, you can pass options to the interpreter by specifying them on the shebang line." That is simply false; if such a thing happens, it's because the interpreter itself is processing the shebang line. If the interpreter has no special recognition for shebang lines, then no such thing happens. Windows doesn't do anything with shebang lines." What you may be describing in this case is the python launcher: python.org/dev/peps/pep-0397. – Clie

Windows has no provision for making a ".py" file executable at all. Python files appear executable from the Explorer shell via an association of the .py suffix as a document to an application. If that application is the Python-specific pylauncher, then you get hash bang processing. That's it. – Clie

@sinanünür - how would a "generic shebang line in one-off scripts" look like that both works for Windows and Linux? – Fortyish

@Fortyish Please read the sentence carefully. It says neither what you or Kaz think it says. For example, perl on Windows does not care one iota that there is no /usr/bin/perl, but will pay attention to the options passed to it. – Robinrobina

@SinanÜnür - you make it sounds like you can write a shebang line #!... that applies universally to both Windows and Linux, e.g. #!/usr/bin/perl ..., but absolute paths will always be different on those platforms. – Fortyish

@Fortyish _Please read the sentence carefully. It says neither what you or Kaz think it says._ – Robinrobina

Answer 3

Expanding a bit on the other answers, here's a little example of how your command line scripts can get into trouble by incautious use of /usr/bin/env shebang lines:

$ /usr/local/bin/python -V
Python 2.6.4

$ /usr/bin/python -V
Python 2.5.1

$ cat my_script.py
#!/usr/bin/env python
import json
print "hello, json"

$ PATH=/usr/local/bin:/usr/bin

$ ./my_script.py
hello, json

$ PATH=/usr/bin:/usr/local/bin

$ ./my_script.py
Traceback (most recent call last):
  File "./my_script.py", line 2, in <module>
    import json
ImportError: No module named json

The json module doesn't exist in Python 2.5.

One way to guard against that kind of problem is to use the versioned python command names that are typically installed with most Pythons:

$ cat my_script.py
#!/usr/bin/env python2.6
import json
print "hello, json"

If you just need to distinguish between Python 2.x and Python 3.x, recent releases of Python 3 also provide a python3 name:

$ cat my_script.py
#!/usr/bin/env python3
import json
print("hello, json")

Peregrine

Hmm, that's not what I got out of that post. – Semibreve

Difference between local and global. If which python returns /usr/bin/python, a local directory path could be hard coded: #!/usr/bin/python. But that is less flexible than #!/usr/bin/env python which has a global application. – Thankyou

Answer 4

In order to run the python script, we need to tell the shell three things:

1. That the file is a script
2. Which interpreter we want to execute the script
3. The path of said interpreter

The shebang #! accomplishes (1.). The shebang begins with a # because the # character is a comment marker in many scripting languages. The contents of the shebang line are therefore automatically ignored by the interpreter.

The env command accomplishes (2.) and (3.). To quote "grawity,"

A common use of the env command is to launch interpreters, by making use of the fact that env will search $PATH for the command it is told to launch. Since the shebang line requires an absolute path to be specified, and since the location of various interpreters (perl, bash, python) may vary a lot, it is common to use:

#!/usr/bin/env perl  instead of trying to guess whether it is /bin/perl, /usr/bin/perl, /usr/local/bin/perl, /usr/local/pkg/perl, /fileserver/usr/bin/perl, or /home/MrDaniel/usr/bin/perl on the user's system...

On the other hand, env is almost always in /usr/bin/env. (Except in cases when it isn't; some systems might use /bin/env, but that's a fairly rare occassion and only happens on non-Linux systems.)

Lawler

Answer 5

The exec system call of the Linux kernel understands shebangs (#!) natively

When you do on bash:

./something

on Linux, this calls the exec system call with the path ./something.

This line of the kernel gets called on the file passed to exec: https://github.com/torvalds/linux/blob/v4.8/fs/binfmt_script.c#L25

if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!'))

It reads the very first bytes of the file, and compares them to #!.

If the comparison is true, then the rest of the line is parsed by the Linux kernel, which makes another exec call with:

• executable: /usr/bin/env
• first argument: python
• second argument: script path

therefore equivalent to:

/usr/bin/env python /path/to/script.py

env is an executable that searches PATH to e.g. find /usr/bin/python, and then finally calls:

/usr/bin/python /path/to/script.py

The Python interpreter does see the #! line in the file, but # is the comment character in Python, so that line just gets ignored as a regular comment.

And yes, you can make an infinite loop with:

printf '#!/a\n' | sudo tee /a
sudo chmod +x /a
/a

Bash recognizes the error:

-bash: /a: /a: bad interpreter: Too many levels of symbolic links

#! just happens to be human readable, but that is not required.

If the file started with different bytes, then the exec system call would use a different handler. The other most important built-in handler is for ELF executable files: https://github.com/torvalds/linux/blob/v4.8/fs/binfmt_elf.c#L1305 which checks for bytes 7f 45 4c 46 (which also happens to be human readable for .ELF). Let's confirm that by reading the 4 first bytes of /bin/ls, which is an ELF executable:

head -c 4 "$(which ls)" | hd 

output:

00000000  7f 45 4c 46                                       |.ELF|
00000004                                                                 

So when the kernel sees those bytes, it takes the ELF file, puts it into memory correctly, and starts a new process with it. See also: How does kernel get an executable binary file running under linux?

Finally, you can add your own shebang handlers with the binfmt_misc mechanism. For example, you can add a custom handler for .jar files. This mechanism even supports handlers by file extension. Another application is to transparently run executables of a different architecture with QEMU.

I don't think POSIX specifies shebangs however: https://unix.stackexchange.com/a/346214/32558 , although it does mention in on rationale sections, and in the form "if executable scripts are supported by the system something may happen". macOS and FreeBSD also seem to implement it however.

PATH search motivation

Likely, one big motivation for the existence of shebangs is the fact that in Linux, we often want to run commands from PATH just as:

basename-of-command

instead of:

/full/path/to/basename-of-command

But then, without the shebang mechanism, how would Linux know how to launch each type of file?

Hardcoding the extension in commands:

 basename-of-command.py

or implementing PATH search on every interpreter:

python basename-of-command

would be a possibility, but this has the major problem that everything breaks if we ever decide to refactor the command into another language.

Shebangs solve this problem beautifully.

Major use case of env: pyenv and other version managers

One major use case of why you should use #!/usr/bin/env python instead of just /usr/bin/python is that of version managers with pyenv.

pyenv allows you to easily install multiple python versions on a single machine, to be able to better reproduce other projects without virtualization.

Then, it manages the "current" python version by setting its order in the PATH: e.g. as shown at apt-get install for different python versions a pyenv managed python could be located at:

/home/ciro/.pyenv/shims/python

so nowhere close to /usr/bin/python, which some systems might deal with via update-alternatives symlinks.

Katharinekatharsis

Answer 6

Perhaps your question is in this sense:

If you want to use: $python myscript.py

You don't need that line at all. The system will call python and then python interpreter will run your script.

But if you intend to use: $./myscript.py

Calling it directly like a normal program or bash script, you need write that line to specify to the system which program use to run it, (and also make it executable with chmod 755)

Ottilie

or you can write python3 myscript.py – Antislavery

This is the only actual answer to the question asked above. I'm not sure why all those folks answering intricacies about how shebangs work without answering the question (why is there a shebang?) get hundreds of upvotes. – Singlecross

Answer 7

The main reason to do this is to make the script portable across operating system environments.

For example, under MinGW, Python scripts use:

#!/c/python3k/python

And under GNU/Linux distribution it is either

#!/usr/local/bin/python

or

#!/usr/bin/python

And under the best commercial Unix software/hardware system of all (OS X), it is:

#!/Applications/MacPython 2.5/python

or on FreeBSD:

#!/usr/local/bin/python

However, all these differences can make the script portable across all by using:

#!/usr/bin/env python

Paroxysm

Under MacOSX, it is also /usr/bin/python. Under Linux, the Python installed by the system is also almost certainly /usr/bin/python (I have never seen anything else and it would make no sense). Note that there might be systems which don't have /usr/bin/env. – Roofdeck

If you're on OSX and use Homebrew and follow their default installation instructions, it'll be under #!/usr/local/bin/python – Dannettedanni

Update for 2018 year: Bare python is not that portable, it's distribution default Python interpreter. Arch Linux defaults to Python 3 for long time and may distributions are thinking about it too because Python 2 is supported only until 2020. – Solange

Answer 8

Technically, in Python, this is just a comment line.

This line is only used if you run the py script from the shell (from the command line). This is know as the "Shebang!", and it is used in various situations, not just with Python scripts.

Here, it instructs the shell to start a specific version of Python (to take care of the rest of the file.

Amphithecium

The shebang is a Unix concept. Might be worth to mention that it works on Windows too if you have installed the Python launcher py.exe. This is part of a standard Python installation. – Notum

Answer 9

It probably makes sense to emphasize one thing that the most have missed, which may prevent immediate understanding. When you type python in terminal you don't normally provide a full path. Instead, the executable is up looked in PATH environment variable. In turn, when you want to execute a Python program directly, /path/to/app.py, one must tell the shell what interpreter to use (via the hashbang, what the other contributors are explaining above).

Hashbang expects full path to an interpreter. Thus to run your Python program directly you have to provide full path to Python binary which varies significantly, especially considering a use of virtualenv. To address portability the trick with /usr/bin/env is used. The latter is originally intended to alter environment in-place and run a command in it. When no alteration is provided it runs the command in current environment, which effectively results in the same PATH lookup which does the trick.

Source from unix stackexchange

Calesta

Answer 10

This is a shell convention that tells the shell which program can execute the script.

#!/usr/bin/env python

resolves to a path to the Python binary.

Scutiform

Answer 11

It's the recommended way, proposed in documentation:

2.2.2. Executable Python Scripts

On BSD’ish Unix systems, Python scripts can be made directly executable, like shell scripts, by putting the line

#! /usr/bin/env python3.2

From 2. Using the Python Interpreter

Lynnalynne

The link is (effectively) broken. – Trix

Answer 12

It just specifies what interpreter you want to use. To understand this, create a file through terminal by doing touch test.py, then type into that file the following:

#!/usr/bin/env python3
print "test"

and do chmod +x test.py to make your script executable. After this when you do ./test.py you should get an error saying:

  File "./test.py", line 2
    print "test"
               ^
SyntaxError: Missing parentheses in call to 'print'

because python3 doesn't supprt the print operator.

Now go ahead and change the first line of your code to:

#!/usr/bin/env python2

and it'll work, printing test to stdout, because python2 supports the print operator. So, now you've learned how to switch between script interpreters.

Pointtopoint

Answer 13

You can try this issue using virtualenv

Here is test.py

#! /usr/bin/env python
import sys
print(sys.version)

Create virtual environments

virtualenv test2.6 -p /usr/bin/python2.6
virtualenv test2.7 -p /usr/bin/python2.7

activate each environment then check the differences

echo $PATH
./test.py

Margit

Answer 14

It seems to me like the files run the same without that line.

If so, then perhaps you're running the Python program on Windows? Windows doesn't use that line—instead, it uses the file-name extension to run the program associated with the file extension.

However in 2011, a "Python launcher" was developed which (to some degree) mimics this Linux behaviour for Windows. This is limited just to choosing which Python interpreter is run — e.g. to select between Python 2 and Python 3 on a system where both are installed. The launcher is optionally installed as py.exe by Python installation, and can be associated with .py files so that the launcher will check that line and in turn launch the specified Python interpreter version.

Transmigrant

He might also be using $ python myscript.py. – Robinrobina

I made the mistake by not having the line and used python script.py, and one day I just did ./myscript.py and everything stopped working, then realizing the system is looking the file as a shell script instead of a python script. – Spell

Answer 15

This is meant as more of historical information than a "real" answer.

Remember that back in the day you had lots of Unix-like operating systems whose designers all had their own notion of where to put stuff, and sometimes didn't include Python, Perl, Bash, or lots of other GNU/Open-source stuff at all.

This was even true of different Linux distributions. On Linux - pre-FHS¹ - you might have Python in /usr/bin/ or /usr/local/bin/. Or it might not have been installed, so you built your own and put it in ~/bin.

Solaris was the worst I ever worked on, partially as the transition from Berkeley Unix to System V. You could wind up with stuff in /usr/, /usr/local/, /usr/ucb/, /opt/, etc. This could make for some really long paths. I have memories of the stuff from Sunfreeware.com installing each package in its own directory, but I can't recall if it symlinked the binaries into /usr/bin/ or not.

Oh, and sometimes /usr/bin/ was on an NFS server².

So the env utility was developed to work around this.

Then you could write #!/bin/env interpreter and as long as the path was proper things had a reasonable chance of running. Of course, reasonable meant (for Python and Perl) that you had also set the appropriate environmental variables. For bash/ksh/zsh it just worked.

This was important because people were passing around shell scripts (like Perl and Python) and if you'd hard coded /usr/bin/python on your Red Hat Linux workstation it was going to break bad on a SGI...well, no, I think IRIX put Python in the right spot. But on a Sparc station it might not run at all.

I miss my Sparc station. But not a lot. Ok, now you've got me trawling around on eBay. Bastages.

---

¹ File-system Hierarchy Standard.

² Yes, and sometimes people still do stuff like that. And no, I did not wear either a turnip OR an onion on my belt.

Englert

Answer 16

If you're running your script in a virtual environment, say venv, then executing which python while working on venv will display the path to the Python interpreter:

~/Envs/venv/bin/python

Note that the name of the virtual environment is embedded in the path to the Python interpreter. Therefore, hardcoding this path in your script will cause two problems:

• If you upload the script to a repository, you're forcing other users to have the same virtual environment name. This is if they identify the problem first.
• You won't be able to run the script across multiple virtual environments even if you had all required packages in other virtual environments.

Therefore, to add to Jonathan's answer, the ideal shebang is #!/usr/bin/env python, not just for portability across OSes but for portability across virtual environments as well!

Varityper

Answer 17

The line #!/bin/bash/python3 or #!/bin/bash/python specifies which python compiler to use. You might have multiple python versions installed. For example,
a.py :

#!/bin/bash/python3
print("Hello World")

is a python3 script, and
b.py :

#!/bin/bash/python
print "Hello World"

is a python 2.x script
In order to run this file ./a.py or ./b.py is used, you need to give the files execution privileges before hand, otherwise executing will lead to Permission denied error.
For giving execution permission,

chmod +x a.py

Tol

/bin/bash/python? That confuses me. – Arianaariane

@KevinC that is the actual path to the python interprete.r – Tol

Perhaps use: #!/usr/bin/env python3. This way, the system will search it's PATH to find python3, it's a prettier method. – Arianaariane

Answer 18

Considering the portability issues between python2 and python3, you should always specify either version unless your program is compatible with both.

Some distributions are shipping python symlinked to python3 for a while now - do not rely on python being python2.

This is emphasized by PEP 394:

In order to tolerate differences across platforms, all new code that needs to invoke the Python interpreter should not specify python, but rather should specify either python2 or python3 (or the more specific python2.x and python3.x versions; see the Migration Notes). This distinction should be made in shebangs, when invoking from a shell script, when invoking via the system() call, or when invoking in any other context.

Coinstantaneous

Answer 19

It tells the interpreter which version of python to run the program with when you have multiple versions of python.

Figment

Answer 20

It allows you to select the executable that you wish to use; which is very handy if perhaps you have multiple python installs, and different modules in each and wish to choose. e.g.

#!/bin/sh
#
# Choose the python we need. Explanation:
# a) '''\' translates to \ in shell, and starts a python multi-line string
# b) "" strings are treated as string concat by python, shell ignores them
# c) "true" command ignores its arguments
# c) exit before the ending ''' so the shell reads no further
# d) reset set docstrings to ignore the multiline comment code
#
"true" '''\'
PREFERRED_PYTHON=/Library/Frameworks/Python.framework/Versions/2.7/bin/python
ALTERNATIVE_PYTHON=/Library/Frameworks/Python.framework/Versions/3.6/bin/python3
FALLBACK_PYTHON=python3

if [ -x $PREFERRED_PYTHON ]; then
    echo Using preferred python $ALTERNATIVE_PYTHON
    exec $PREFERRED_PYTHON "$0" "$@"
elif [ -x $ALTERNATIVE_PYTHON ]; then
    echo Using alternative python $ALTERNATIVE_PYTHON
    exec $ALTERNATIVE_PYTHON "$0" "$@"
else
    echo Using fallback python $FALLBACK_PYTHON
    exec python3 "$0" "$@"
fi
exit 127
'''

__doc__ = """What this file does"""
print(__doc__)
import platform
print(platform.python_version())

Linville

Answer 21

When you execute the python file, you can use ./file.py where file is the name of the file. /usr/bin/env is the PATH, then python is python 2 and python3 is python 3 (duh)

#!/usr/bin/env python can also allow the python file to be executed by other programs, as long as you use chmod +x file.py.

Bovine

Answer 22

this tells the script where is python directory !

#! /usr/bin/env python

Uticas