Dependency management: subprocess32 needed for Python2.7

Question

I have a library (subx) which depends on subprocess32. The subprocess32 library is a backport for Python2.7 and provides the timeout kwarg.

My library needs the timeout kwarg.

I need subprocess32 only if the target platform is Python2.x.

How should I define the dependency in my project?

I get this error message, if I define a dependency to subprocess32 via "install_requires" (setup.py) and I am inside a python3 virtualenv:

===> pip install -e git+https://github.com/guettli/subx.git#egg=subx
Obtaining subx from git+https://github.com/guettli/subx.git#egg=subx
  Cloning https://github.com/guettli/subx.git to ./src/subx
Collecting subprocess32 (from subx)
  Using cached subprocess32-3.2.7.tar.gz
    Complete output from command python setup.py egg_info:
    This backport is for Python 2.x only.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-lju3nl1y/subprocess32/

Elainaelaine

Answer 1

There is a declarative way, but afaik it requires a more or less recent version of setuptools (if I read the release notes correctly, you need at least the 20.2 version). What you will see down below are called environment markers and are specified in PEP 508, read it through for a full list of available markers and a better understanding of the marker syntax if you wish.

For python version, let's take your package as an example: you have subprocess32 dependency that should be installed in python2.X environment. Enhance your dependency like this:

install_requires=[
    'subprocess32; python_version<"3"',
]

Installing the package subx with python2.7 now yields:

Processing ./dist/subx-2017.8.0-py2-none-any.whl
Collecting subprocess32; python_version < "3" (from subx==2017.8.0)
Installing collected packages: subprocess32, subx
Successfully installed subprocess32-3.2.7 subx-2017.8.0

If you install it with python3.X, the output will be:

Processing ./dist/subx-2017.8.0-py3-none-any.whl
Installing collected packages: subx
Successfully installed subx-2017.8.0

Notice that the installation of subprocess32 will be skipped.

---

Another common example is to declare platform-specific dependencies: I have a project that requires auditwheel to be installed on Linux and delocate on MacOS. I declare the dependencies like this:

install_requires=[
    ...
    'auditwheel==1.7.0; "linux" in sys_platform',
    'delocate==0.7.1; "darwin" == sys_platform',
]

Note that this check for Linux is needed if you do not specifically target any major python version because:

$ python2 -c "import sys; print sys.platform"
linux2

but

$ python3 -c "import sys; print sys.platform"
linux

so if for example your package works only with python2.X, you can use the check "linux2" == sys.platform. This will make your dependency installable only with python2.X.

However

This seems a complete answer. However, in windows sys.platform will print win32 for Windows and cygwin for Windows/Cygwin. – Floris

@ElisByberi afaik basic expressions are supported so "win32" == sys_platform or "cygwin" == sys_platform should work. However, there are other environment markers like platform_system that would probably better suit this case. Again, check PEP 508 for further details. – However

Answer 2

import sys

kw = {}
if sys.version_info[0] == 2:
    kw['install_requires'] = ['subprocess32']

setup(
    …
    **kw
)

Nesta

It would be highly recommended to add some explanations of how your code operates. Simply providing code is not the best method to learning programming, and will not benefit the post creator as much as it would if it were explained well. A well-detailed answer will establish your credibility in the community, and your answers will attract up-votes. – Gingili

Yes, works. But I ask myself if there is a more declarative solution available. This looks like programming, not defining dependencies. Don't get me wrong. Thank you for your answer! – Elainaelaine

This approach will work with source tars only, but fail with wheels. – However

Don't create universal wheel. Version-specific wheels will be ok. – Nesta

Sorry, you're right - only universal wheels are affected. – However