Is there a way to have a conditional requirements.txt file for my Python application based on platform?

Question

I have a python application that I wrote to be compatible with both, Linux and Windows platforms. However there is one problem... One of the python packages I need for Windows is not compatible with Linux. Fortunately there is another package that provides the same functionality on Linux. All other dependencies are compatible in both platforms.

I know I could have 2 separate requirement files to address both platform dependencies separately. Something like win_requirements.txt and linux_requirements.txt, however this approach doesn't feel like the best way to do it.

I wonder if there is a way I can have only one requirements.txt file so any user can use pip install -r requirements.txt to install all the dependencies regardless of what platform they are?

Maybe something like??:

SOAPpy>=0.12.22
pycrypto>=2.6.1
suds>=0.4
Python-ldap>=2.4.19
paramiko>=1.15.2
nose>=1.3.4
selenium>=2.44.0
bottle>=0.12.8
CherryPy>=3.6.0
pika>=0.9.14
if platform.system() == 'Linux':
    wmi-client-wrapper>=0.0.12
else if platform.system() == 'Windows':
    WMI>=1.4.9

Devine

Strongly recommend looking at Tony G's answer instead of the accepted answer: https://mcmap.net/q/25380/-is-there-a-way-to-have-a-conditional-requirements-txt-file-for-my-python-application-based-on-platform – Orestes

Answer 1

You could create an install.py script and call pip by script.

import pip

_all_ = [
    "SOAPpy>=0.12.22",
    "pycrypto>=2.6.1",
    "suds>=0.4",
    "Python-ldap>=2.4.19",
    "paramiko>=1.15.2",
    "nose>=1.3.4",
    "selenium>=2.44.0",
    "bottle>=0.12.8",
    "CherryPy>=3.6.0",
    "pika>=0.9.14",
]

windows = ["wmi-client-wrapper>=0.0.12",]

linux = ["WMI>=1.4.9",]

darwin = []

def install(packages):
    for package in packages:
        pip.main(['install', package])

if __name__ == '__main__':

    from sys import platform

    install(_all_) 
    if platform == 'windows':
        install(windows)
    if platform.startswith('linux'):
        install(linux)
    if platform == 'darwin': # MacOS
        install(darwin)

Another way to resolve this issue using only requirements files should be using inheritance of requirements

requirements.txt

SOAPpy>=0.12.22
pycrypto>=2.6.1
suds>=0.4
Python-ldap>=2.4.19
paramiko>=1.15.2
nose>=1.3.4
selenium>=2.44.0
bottle>=0.12.8
CherryPy>=3.6.0

windows.txt

-r requirements.txt
WMI>=1.4.9

linux.txt

-r requirements.txt
WMI>=1.4.9

Then you can call just the requirement equivalent to platform.

pip install -r windows.txt
pip install -r linux.txt

Glochidiate

In additon, one may want to keep the install.py cleaner by replacing the all block with a base_requirements.txt file and using -r base_requirements.txt at the top of the different requirements files. – Soapy

pip.main() is deprecated now. See https://mcmap.net/q/20296/-how-can-i-install-a-python-module-within-code – Concordance

if you have '-r requirements.txt', and requirements has 'WMI>=1.4.9', while windows.txt has an earlier version 'WMI>=1.4.8', will it down grade the version? – Whilom

@Whilom take a look here – Glochidiate

Answer 2

You can add certain conditional requirements after a semi-colon. Particularly useful for sys_platform and python_version.

Examples:

atomac==1.1.0; sys_platform == 'darwin'
futures>=3.0.5; python_version < '3.0'
futures>=3.0.5; python_version == '2.6' or python_version=='2.7'

Apparently you can also exclude particular versions of a library:

futures>=3.0,!=3.0.5

They are defined in PEP 508 and PEP 0345 (Environment Markers) but the syntax appears to follow the draft PEP 0496.

Whelm

is it possible to query for pypy too? – Pristine

Actually, I believe its PEP 508. – Servant

@ sebix yes have a look at the PEP. – Servant

Does not seem to work with hash pinning: if I have the line enum34==1.1.6 --hash=sha256:644837f692e5f550741432dd3f223bbb9852018674981b1664e5dc339387588a; python_version < '3.4' I get Expected sha256 644837f692e5f550741432dd3f223bbb9852018674981b1664e5dc339387588a;, Got 644837f692e5f550741432dd3f223bbb9852018674981b1664e5dc339387588a, if I add a space before the ;, the version condition is just ignored. – Countless

Would there be a way to check if a user uses ssh or https to install packages from git? And depending on that provide a different link to install a package from a private git repo? Or do something like try except? – Licensee

@SimoneBronzini try : enum34==1.1.6; python_version < '3.4' --hash=sha256:644837f692e5f550741432dd3f223bbb9852018674981b1664e5dc339387588a – Caelian

For PyPy: platform_python_implementation == "PyPy" – Formula

Answer 3

You can add additional requirements to any package after a semicolon. You may limit any package with multi-condition by and, or. more conditions: https://www.python.org/dev/peps/pep-0508/#environment-markers

examples:

futures>=3.0.5; python_version < '3.0'
futures>=3.0.5; python_version == '2.6' or python_version=='2.7'
futures>3 ; python_version >= "3.6" and sys_platform == "linux"
futures>3.3 ; python_version >= "3.6" and sys_platform == "darwin"

Shanika

Thanks, the enivironment markers helped. I needed to use os_name to check if it is raspberrypi – Notorious

Answer 4

Use this in the requirements.txt file

uwsgi==2.0.18; platform_system != "Windows"

in this case pip will install uwsgi if not running on Windows

Emulsion

Answer 5

A Complimentary questions though, What if none of the default markers is suitable for our usecase. For example, if I want to install a package when it is run in CI vs when it runs on my machine , Can I set an environment variable and decideds based on its value ?

Mozza