DistutilsOptionError: must supply either home or prefix/exec-prefix -- not both

Question

I've been usually installed python packages through pip.

For Google App Engine, I need to install packages to another target directory.

I've tried:

pip install -I flask-restful --target ./lib

but it fails with:

must supply either home or prefix/exec-prefix -- not both

How can I get this to work?

Decathlon

Answer 1

Are you using OS X and Homebrew? The Homebrew python page https://github.com/Homebrew/brew/blob/master/docs/Homebrew-and-Python.md calls out a known issue with pip and a work around.

Worked for me.

You can make this "empty prefix" the default by adding a ~/.pydistutils.cfg file with the following contents:

[install]
prefix=

Edit: The Homebrew page was later changed to recommend passing --prefix on the command line, as discussed in the comments below. Here is the last version which contained that text. Unfortunately this only works for sdists, not wheels.

The issue was reported to pip, which later fixed it for --user. That's probably why the section has now been removed from the Homebrew page. However, the problem still occurs when using --target as in the question above.

Eyed

good stuff, the link is bad, this is the new one I expect : github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/… – Ebbie

Note that that file broke the virtual environment for me. – Territorialize

As Dmitry says, adding the ~/.pydistutils.cfg file fixes the original problem, but causes the virtualenv command to break. I worked around it by renaming the file when I need to run virtualenv but obviously this is not ideal. – Ferment

This empty prefix business breaks normal pip install operations :( – Loesch

Found some interesting discussion here: #2915971 Haven't tried it yet, but using --install-option="--prefix=lib" might be the way to go. – Mcbroom

No dice. OSError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/argparse-1.3.0.dist-info/DESCRIPTION.rst'. – Mcbroom

has anyone figured out how to allow --target while not ruining default pip install behavior? – Mulry

Yes, this breaks pip operations. I sent a PR to Homebrew to remove this recommendation github.com/Homebrew/brew/pull/821 – Monmouthshire

This doesn't appear to be valid anymore. The link is broken and the updated link doesn't talk about pydistutils.cfg – Roband

Why doesn't this work when you specify --prefix '' instead of creating a file? – Justiciar

Answer 2

I believe there is a simpler solution to this problem (Homebrew's Python on macOS) that won't break your normal pip operations.

All you have to do is to create a setup.cfg file at the root directory of your project, usually where your main __init__.py or executable py file is. So if the root folder of your project is: /path/to/my/project/, create a setup.cfg file in there and put the magic words inside:

[install]
prefix=  

OK, now you sould be able to run pip's commands for that folder:

pip install package -t /path/to/my/project/  

This command will run gracefully for that folder only. Just copy setup.cfg to whatever other projects you might have. No need to write a .pydistutils.cfg on your home directory.

After you are done installing the modules, you may remove setup.cfg.

Ophthalmic

This worked perfectly with pip3.6 as well. And my pip is still intact. – Zindman

This should be the accepted answer. Avoids causing issues with global pip settings. – Plafond

emphasis on the remove setup.cfg after installation part. I burnt thru 2 whole days trying to figure out why my virtualenv environment was screwed, with errors like Could not install packages due to an EnvironmentError: [Errno 1] Operation not permitted: '/bin/easy_install'. Removing the setup file restored my sanity – Crompton

@Crompton yes rightly noted by you, so I have edited the answer to emphasise that bit, AndreG please note. – Britteny

Thanks for noting. Would it be more correct if I said "After you are done installing the modules, you should remove setup.cfg"? – Ophthalmic

I think, we could say, "... you may/should remove ... until you need to install more modules". Because there might be more modules in the future during the course of development. Rest is your call AndreG. – Britteny

Let's keep it this way with your emphasis then. – Ophthalmic

On "to this problem (Homebrew's Python on macOS)": is there an issue registered somewhere that we could track to see when/if this is solved? I could find github.com/Homebrew/brew/issues/4165, but there could be more. – Fic

Answer 3

On OSX(mac), assuming a project folder called /var/myproject

1. cd /var/myproject
2. Create a file called setup.cfg and add [install] prefix=
3. Run pip install <packagename> -t .

Partisan

I'm not sure how this is different from @Ophthalmic answer – Mockheroic

The difference for me is that this solution cd's into the directory and does -t . instead of staying outside the dir. This way worked for me and the other did not, though I have no idea why. – Bolling

Answer 4

Another solution* for Homebrew users is simply to use a virtualenv.

Of course, that may remove the need for the target directory anyway - but even if it doesn't, I've found --target works by default (as in, without creating/modifying a config file) when in a virtual environment.

---

*I say solution; perhaps it's just another motivation to meticulously use venvs...

Venue

Can't believe this was only a month ago. Just caught myself answering my own question; ahead of time... – Venue

So, I recently had the problem in OP's question, and merely creating a virtualenv solved the issue for me. I could still install into the target directory. My issue was additionally complicated because i installed python3 as well, but +1 for the virtualenv solution. – Standardize

Answer 5

I hit errors with the other recommendations around --install-option="--prefix=lib". The only thing I found that worked is using PYTHONUSERBASE as described here.

export PYTHONUSERBASE=lib
pip install -I flask-restful --user

this is not exactly the same as --target, but it does the trick for me in any case.

Astra

Answer 6

As other mentioned, this is known bug with pip & python installed with homebrew.

If you create ~/.pydistutils.cfg file with "empty prefix" instruction it will fix this problem but it will break normal pip operations.

Until this bug is officially addressed, one of the options would be to create your own bash script that would handle this case:

 #!/bin/bash

 name=''
 target=''

 while getopts 'n:t:' flag; do
     case "${flag}" in
         n) name="${OPTARG}" ;;
         t) target="${OPTARG}" ;;
     esac
 done

 if [ -z "$target" ];
 then
     echo "Target parameter must be provided"
     exit 1
 fi

 if [ -z "$name" ];
 then
     echo "Name parameter must be provided"
     exit 1
 fi

 # current workaround for homebrew bug
 file=$HOME'/.pydistutils.cfg'
 touch $file

 /bin/cat <<EOM >$file
 [install]
 prefix=
 EOM
 # end of current workaround for homebrew bug

 pip install -I $name --target $target

 # current workaround for homebrew bug
 rm -rf $file
 # end of current workaround for homebrew bug

This script wraps your command and:

1. accepts name and target parameters
2. checks if those parameters are empty
3. creates ~/.pydistutils.cfg file with "empty prefix" instruction in it
4. executes your pip command with provided parameters
5. removes ~/.pydistutils.cfg file

This script can be changed and adapted to address your needs but you get idea. And it allows you to run your command without braking pip. Hope it helps :)

Willful

Answer 7

If you're using virtualenv*, it might be a good idea to double check which pip you're using.

If you see something like /usr/local/bin/pip you've broken out of your environment. Reactivating your virtualenv will fix this:

VirtualEnv: $ source bin/activate

VirtualFish: $ vf activate [environ]

*: I use virtualfish, but I assume this tip is relevant to both.

Guarded

using virtualenv was actually the solution in my similar case :) – Muscular

Answer 8

I have a similar issue. I use the --system flag to avoid the error as I decribe here on other thread where I explain the specific case of my situation. I post this here expecting that can help anyone facing the same problem.

Esquivel