I want to get the dependency of a PyPi package remotely without needing to download it completely.
I seem to understand (reading the pip code) that pip when resolving dependencies seems to read the egg once the package has been downloaded.
Is there any other way?
I've just needed to find a way to do this and this is what I came up with (stolen from pip).
def dist_metadata(setup_py):
'''Get the dist object for setup.py file'''
with open(setup_py) as f:
d = f.read()
try:
# we have to do this with current globals else
# imports will fail. secure? not really. A
# problem? not really if your setup.py sources are
# trusted
exec d in globals(), globals()
except SystemExit:
pass
return distutils.core._setup_distribution
https://mcmap.net/q/416021/-why-doesn-39-t-an-import-in-an-exec-in-a-function-work answers why the exec incantation is subtle and hard to get right.
import distutils at the top, and the caller will want to look at dist_metadata(setup_py).install_requires, which returns a list of package names. That leaves the question of how to get the setup.py without downloading the whole package, as OP requested. –
Alessandro exec d in globals(), globals() to exec(d, globals()) and to get requirements looks at the dist_metadata(setup_py).get_requires() method. –
Elaelaborate As jinghli notes, there isn't currently a reliable way to get the dependency of an arbitrary PyPi package remotely without needing to download it completely. And in fact the dependencies sometimes depend on your environment, so an approach like Brian's of executing setup.py code is needed in the general case.
The way the Python ecosystem handles dependencies started evolving in the 1990's before the problem was well understood. PEP 508 -- Dependency specification for Python Software Packages sets us on course to improve the situtation, and an "aspirational" draft approach in PEP 426 -- Metadata for Python Software Packages 2.0 may improve it more in the future, in conjunction with the reimplementation of PyPI as Warehouse.
The current situation is described well in the document Python Dependency Resolution.
PyPI does provide a json interface to download metadata for each package. The info.requires_dist object contains a list of names of required packages with optional version restrictions etc. It is often missing, but it is one place to start.
E.g. Django (json) indicates:
{
"info": {
...
"requires_dist": [
"bcrypt; extra == 'bcrypt'",
"argon2-cffi (>=16.1.0); extra == 'argon2'",
"pytz"
],
...
}
requires_dist block doesn't show up for the linked JSON from pypi anymore :( –
Rambert requires_dist –
Alessandro requires_dist. This answer should be the accepted one, nice explanation @nealmcb! –
Anhydrous Use pipdeptree to view dependencies of installed PyPI packages.
Install:
pip install pipdeptree
Then run:
pipdeptree
You'll see something like that:
Warning!!! Possible conflicting dependencies found:
* Mako==0.9.1 -> MarkupSafe [required: >=0.9.2, installed: 0.18]
Jinja2==2.7.2 -> MarkupSafe [installed: 0.18]
------------------------------------------------------------------------
Lookupy==0.1
wsgiref==0.1.2
argparse==1.2.1
psycopg2==2.5.2
Flask-Script==0.6.6
- Flask [installed: 0.10.1]
- Werkzeug [required: >=0.7, installed: 0.9.4]
- Jinja2 [required: >=2.4, installed: 2.7.2]
- MarkupSafe [installed: 0.18]
- itsdangerous [required: >=0.21, installed: 0.23]
alembic==0.6.2
- SQLAlchemy [required: >=0.7.3, installed: 0.9.1]
- Mako [installed: 0.9.1]
- MarkupSafe [required: >=0.9.2, installed: 0.18]
ipython==2.0.0
slugify==0.0.1
redis==2.9.1
I've just needed to find a way to do this and this is what I came up with (stolen from pip).
def dist_metadata(setup_py):
'''Get the dist object for setup.py file'''
with open(setup_py) as f:
d = f.read()
try:
# we have to do this with current globals else
# imports will fail. secure? not really. A
# problem? not really if your setup.py sources are
# trusted
exec d in globals(), globals()
except SystemExit:
pass
return distutils.core._setup_distribution
https://mcmap.net/q/416021/-why-doesn-39-t-an-import-in-an-exec-in-a-function-work answers why the exec incantation is subtle and hard to get right.
import distutils at the top, and the caller will want to look at dist_metadata(setup_py).install_requires, which returns a list of package names. That leaves the question of how to get the setup.py without downloading the whole package, as OP requested. –
Alessandro exec d in globals(), globals() to exec(d, globals()) and to get requirements looks at the dist_metadata(setup_py).get_requires() method. –
Elaelaborate You can fetch PyPI package metadata in JSON format using a URL like: https://pypi.org/pypi/<package name>/json.
Example: https://pypi.org/pypi/tensorflow/json
From a comment on the OP that was the right answer for me.
Sadly, pip doesn't have this function. The metadata available for packages on PyPI does not include information about the dependencies.
Normally, you can find the detailed dependency from the README file from project website.
pip search can give some information about the package. It can tell you what is it based on.
$ pip search flask
Flask - A microframework based on Werkzeug, Jinja2 and good intentions
© 2022 - 2024 — McMap. All rights reserved.
requirements.txtor insetup.py, maybe you could download just a single file from the repo, depending on where it's hosted? (i.e. github as opposed to PyPi) Related Q – Christianachristianehttps://pypi.org/pypi/<package name>/json, for example pypi.org/pypi/tensorflow/json – Margueritehttps://pypi.org/pypi/<package name>/<package version>/jsonadditionally, for example pypi.org/pypi/pyqt5-tools/5.15.4.3.2/json. – Housesetup.py, for example github.com/altendky/pyqt-tools/blob/main/setup.py#L88. – House