Troubles while installing psutil (wheel) as a dependency via pip

Question

I wrote a package with a dependency's dependency to psutil (my-package depends on third-party-package which depends on psutil).
Since it's supposed to run on a server without any connectivity and without gcc, I prepared the deployment locally with a psutil python platform wheel and pip install my-package --download, then sent everything on the server.

Now everything is ready on the server, but for some reason, when I run the installation, pip refuses to install psutil. Note that the server is a red hat 7.2 running pip 7.1.0, virtualenv 1.10.1 and python 2.7.5 (and I can't change the version of anything).

$ pip install /tmp/python_packages/my-package-1.4.zip --no-index 
  --find-links /tmp/python_packages/ --use-wheel
Ignoring indexes: https://pypi.python.org/simple/

# blablabla, everything goes fine, then

Downloading/unpacking psutil (from third-party-package>=0.9->my-package==1.4)
  Could not find any downloads that satisfy the requirement psutil 
  (from third-party-package>=0.9->my-package==1.4)
Cleaning up...
No distributions at all found for psutil (from third-party-package>=0.9->my-package==1.4)
Storing complete log in /home/anto/.pip/pip.log

Here is what pip.log says:

Downloading/unpacking psutil (from third-party-package>=0.9->my-package==1.4)

  URLs to search for versions for psutil (from third-party-package>=0.9->my-package==1.4):
  # bla
  Skipping file:///tmp/python_packages/psutil-4.2.0-cp27-cp27mu-linux_x86_64.whl 
    because it is not compatible with this Python
  # bla
  Could not find any downloads that satisfy the requirement psutil (from third-
   party-package>=0.9->my-package==1.4)

Cleaning up...

So "not compatible with this Python", ok. But here is the really weird part: if I install psutil without anything else, and then the rest, everything goes fine.

$ pip install /tmp/python_packages/psutil-4.2.0-cp27-cp27mu-linux_x86_64.whl 
Unpacking /tmp/python_packages/psutil-4.2.0-cp27-cp27mu-linux_x86_64.whl
Installing collected packages: psutil
Successfully installed psutil
Cleaning up...
$ pip freeze -l
psutil==4.2.0
$ pip install /tmp/python_packages/my-package-1.4.zip --no-index 
  --find-links /tmp/python_packages/ --use-wheel

# blablabla

Successfully installed my-package third-party-package
Cleaning up...

What am I missing ? Any clue ?

Straight

what's the third party package? – Untie

It's a package developed by someone within my company. The setup.py contains install_requires = ['psutil']; nothing fancy. – Straight

Not sure if you have multiple python versions, but in the pip.log verify that in both cases that the python used is the correct version? (#29880004) Not convinced that is your problem, but is a start. On another note, why are you still on 2.7.5? latest is 2.7.11. – Reportage

Also, it looks like your psutil is 64bit maybe that is part of the issue? – Reportage

Hello, thanks for your suggestions. I think the python version is consistent on the system and pip, but I'll double check. I'm not using the latest python because this server is shipped with 2.7.5 and I'm stuck with it. psutil's wheel is 64bits because the VM is 64bits (but perhaps it could be related to the root cause indeed ?) – Straight

Which version of pip, wheel and setuptools do you use? – Rheims

Answer 1

Make sure that the version of wheel you use to create the wheel is the same as the version used during deployment.

I experienced the same problem when trying to deploy a wheel built with 0.29.0, using wheel 0.24.0.

I downgraded the version of wheel used to build the wheel to match the version used during deployment, and this resolved the problem.

Chappell

Answer 2

From your answer, it seems that you got it working with the extra step.

When doing an offline install, you need to tell it where to find all the packages you want to install explicitly, not just the top level.

I personally have always been explicit about the fact you are looking in the file, and like any other pip install you can specify multiple installs on the same line for context.

$ pip install --no-index --find-links file:/tmp/python_packages/ --use-wheel psutil my-package-1.4.zip

Ochone

I don't think this is the reason because pip finds the wheel; it just refuses to install it for a reason that doesn't sound legit (python version) ;) – Straight

This has more to do with the pip/ven/setuptools version than the Python version IMHO. @Anto: what is the version of these tools you use? And for the sake of it which Linux distro/version? – Rheims

What is the output of pip --version? – Ochone

Answer 3

The key issue you are facing is IMHO this:

Skipping file:///tmp/python_packages/psutil-4.2.0-cp27-cp27mu-linux_x86_64.whl 
because it is not compatible with this Python

A new recent feature has been introduced in pip to support "many linux" wheels (See this PEP)

When asking for a direct install of wheel, minimal checks are done and pip assumes you really want this installed.

When a wheel is installed indirectly as a dep of a dep in your case, there may be incompatible tags in this context.

I would ensure that I use the latest version of pip, setuptools and virtualenv to remove moving parts and it may fix you problem.

Vaguely related, and if this can help, I use this script with a Linux/Mac or Windows wrapper to perform consistent install of vendored pip packages. The key point is to vendor everything and this may be another reason why you get some failure.

Rheims