"WHY" 2 different executables of python of same version?

Question

When I press tab on the command python3.7 I get the following

python3.7          python3.7-config   python3.7m         python3.7m-config

I looked up what's python3.7m and found the answer - https://mcmap.net/q/158364/-difference-between-python3-and-python3m-executables.

Next I go to the python terminal of each implementation and type the following code

>>> import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')

I get the same output in both python implementations i.e .cpython-37m-darwin.so

I also tried the command diff <(python3.7 -m sysconfig) <(python3.7m -m sysconfig) to see if there's any difference in configuration info of the 2 executables but the output is empty means they are the same.

If all the executables and configuration variables are same, then why create two different implementations of python?

Note:

I'm not talking about python3.7/3.7m-config here.

Lewis

python3.7-config is not the same executable. According to the man page "python-config helps compiling and linking programs, which embed the Python interpreter, or extension modules that can be loaded dynamically (at run time) into the interpreter". As for python3.7 and python3.7m, I wasn't aware of the "-m" version, but in Linux it's pretty common to have symlinks for one application to another. For example, changes are that there also is a python3 command at your system that points to the python3.7 executable. – Barcarole

@Barcarole I'm not asking about python3.7-config. – Lewis

You mentioned it in the first few lines. Maybe you should remove that if it's not relevant to your question. – Barcarole

I don't know the answer, but that PEP seems to have a hint: "By default in Python 3.2, configure enables --with-pymalloc so shared library file names would appear as foo.cpython-32m.so". So maybe python3m is kept around for compatibility? – Jehoash

Answer 1

Binary compatibility with C extensions

For a C extension to be compatible across interpreter builds, some of the build-time flags need to be identical. One of these flags is whether pymalloc (a memory-allocation library intended to be faster for Python-specific use cases) is enabled. Versions of libpython have an m suffix indicating that this flag was present at build time, when it in fact was.

If a C extension is built with pymalloc support, it can't be used by an interpreter using the standard C library's malloc(), and the inverse. Consequently, for someone who has a C extension as a prebuilt binary that was built with pymalloc, it can be useful to be able to start a Python interpreter that is known to be itself pymalloc-enabled.

If a Linux distribution chooses to only ship a pymalloc-enabled interpreter, it makes sense to have only one binary available under both python3.7 and python3.7m names. (This leaves folks who need a non-pymalloc interpreter a bit out of luck, but build tools aren't as hard to come by on Linux as they are on Windows, so ABI compatibility is not as critical).

Diannediannne

Answer 2

python3.7 and python3.7m are the same program, just with two different names. These two files are hard-linked, meaning they point to the same file on disk (i.e., they have the same inode).

Here is the line in the cpython 3.7 Makefile that performs this hardlink.

(cd $(DESTDIR)$(BINDIR); $(LN) python$(LDVERSION)$(EXE) python$(VERSION)$(EXE));

$(LDVERSION) would be 3.7m, and $(VERSION) would be 3.7. This is the only place in the Makefile that performs a hardlink.

---

The python3.7 Docker image is used below to demonstrate that python3.7 and python3.7 have the same inode.

$ docker run --rm -it python:3.7-alpine ash
/ # ls -i $(which python3.7)
 927902 /usr/local/bin/python3.7
/ # ls -i $(which python3.7m)
 927902 /usr/local/bin/python3.7m

Dela

Thanks, if you read my question, it asks WHY 2 different executables – Lewis

@bigbounty, the answer already covers it. They're not two different executables. Being hardlinked together means they're two names for the same executable, but it's only actually stored on your hard drive once. – Diannediannne

@bigbounty, ...in this case, that makes sense because the m suffix implies compilation with pymalloc, and so means that the interpreter can be used with C extensions compiled for pymalloc-based interpreters (that's a big deal in the Windows world where most users don't have compilers and can't compile extensions themselves, so they need to install versions built for the interpreter type they actually have). If your main Python interpreter is the same as your pymalloc-enabled Python interpreter, it makes sense to have both names point to the same place. – Diannediannne