pip install failing with: OSError: [Errno 13] Permission denied on directory

Question

pip install -r requirements.txt fails with the exception below OSError: [Errno 13] Permission denied: '/usr/local/lib/.... What's wrong and how do I fix this? (I am trying to setup Django)

Installing collected packages: amqp, anyjson, arrow, beautifulsoup4, billiard, boto, braintree, celery, cffi, cryptography, Django, django-bower, django-braces, django-celery, django-crispy-forms, django-debug-toolbar, django-disqus, django-embed-video, django-filter, django-merchant, django-pagination, django-payments, django-storages, django-vote, django-wysiwyg-redactor, easy-thumbnails, enum34, gnureadline, idna, ipaddress, ipython, kombu, mock, names, ndg-httpsclient, Pillow, pyasn1, pycparser, pycrypto, PyJWT, pyOpenSSL, python-dateutil, pytz, requests, six, sqlparse, stripe, suds-jurko
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1436, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 672, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 902, in move_wheel_files
    pycompile=self.pycompile,
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 206, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 193, in clobber
    os.makedirs(destsubdir)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/amqp-1.4.6.dist-info'

Habitant

Related (macOS/homebrew specific) #33005208 – Bodiless

Answer 1

Option a) Create a virtualenv, activate it and install:

virtualenv .venv
source .venv/bin/activate
pip install -r requirements.txt

Option b) Install in your homedir:

pip install --user -r requirements.txt

My recommendation use safe (a) option, so that requirements of this project do not interfere with other projects requirements.

Cioban

I got an error like this sudo:pip: command not found on my aws ec2 instance when running this command. Please help. – Salas

@Salas Likely your version of pip is in /usr/local/bin which is not on root user's PATH. You can try providing the absolute path /usr/local/bin/pip in the command above. – Blah

@Salas Probably, pip is not installed by default. Which distro is your EC2? Also, python2 might not be installed, so either you install python2 or use pip3. Be careful with this though. – Cioban

I've read this is not recommended in multiple places now. Seems like we should caution against sudo usage when running pip (see Bert's answer) – Contradictory

@JustusEapen: I don't know how I feel about that. I don't think the proper answer to OP's question is a manual on basic computer hygiene, including "don't run shady code with superuser permissions" and "brush your teeth regularly". I find the optimal answer should point out that packages can be installed on a per-user or system-wide basis, and that installing system-wide, as OP wished (there are perfectly cromulent reasons to do so) requires super user permission. Cautioning against installing packages on the system path is probably some else's job on some other SO question. – Lazaretto

I think @bert's answer to use --user argument is more appropriate than sudo unless you actually need to install it globally for all users – Alithia

downvoting because of sudo advise. even though it works now, it is going to give you a lot of headaches in the future. – Uncanonical

sudo will not always work, and is more like using a cheat code. Should mention --user and explain more about use of each. – Tradescantia

pip install --user worked out for me..I personally don't like encouraging the use of sudo unnecessarily – Photoperiod

@Uncanonical will it? It depends entirely on whether OP wants to install Django on the systemwise python path or on a per-user basis, doesn't it? It's blatantly obvious to anybody that installing anything from a repository on the system path requires a certain amount of trust. I don't see the need to downvote. It's not the answerer's duty to give OP the "basic security 101" lecture, since OP likely knows it perfectly well. If anything, it's OP duty to be more specific. @DanCiborowski: I don't see how sudo is a "cheat code". You need to achieve su privileges somehow to alter the system. – Lazaretto

I am getting the error on the virtual env: has anyone? – Mange

Answer 2

Rather than using sudo with pip install, It's better to first try pip install --user. If this fails then take a look at the top post here.

The reason you shouldn't use sudo is as follows:

When you run pip with sudo, you are running arbitrary Python code from the Internet as a root user, which is quite a big security risk. If someone puts up a malicious project on PyPI and you install it, you give an attacker root access to your machine.

Superphosphate

Good observation. That, after all, goes for all sudo x install, for all x (including x = make). – Lazaretto

You answer really helped me. I have no write access to the specific directory and cannot use sudo. The only working method is yours. Thanks! – Eisinger

This also solved my problem. What does adding --user do? – Taxicab

@MilesJohnson Adding --user installs the package in your home directory, rather than the root. Installing something to this location doesn't require any extra privileges. – Superphosphate

This! This! 1000 times this! Not only is this a safer approach to pip in general it is a solution to many of the errors found in this answer about upgrading packages. – Ascarid

Additionally, if you are on a remote server behind a proxy, "sudo" prevents you from fetching the packages from internet repositories and/or git repositories of the remote server's network. – Tortoiseshell

All mention of sudo was removed a year ago. This answer is obsolete - please revise and update it. You also need to mention per-user vs system-wide installs, and permissions. Don't use your answer to directly criticize other answers, that will tend to obsolete quickly. – Cantrell

@Cantrell My answer does more than just “directly criticise” the accepted answer. It explains exactly why you should not use sudo and such an explanation is still valid even if the original answer has been edited. Furthermore, on top of explaining what you should not do, it also explains what you should do and so it still makes sense without that context. – Superphosphate

Answer 3

You are trying to install a package on the system-wide path without having the permission to do so.

1. In general, you can use sudo to temporarily obtain superuser permissions at your responsibility in order to install the package on the system-wide path:

    sudo pip install -r requirements.txt
   

   Find more about sudo here.

   Actually, this is a bad idea and there's no good use case for it, see @wim's comment.

2. If you don't want to make system-wide changes, you can install the package on your per-user path using the --user flag.

   All it takes is:

    pip install --user runloop requirements.txt
   

3. Finally, for even finer grained control, you can also use a virtualenv, which might be the superior solution for a development environment, especially if you are working on multiple projects and want to keep track of each one's dependencies.

   After activating your virtualenv with

   $ my-virtualenv/bin/activate

   the following command will install the package inside the virtualenv (and not on the system-wide path):

   pip install -r requirements.txt

Lazaretto

Running pip with root comes with security risks – Bayly

Running anything that runs code from the Internet as root comes with security risks. – Lazaretto

This is nearly the best answer, but still needs updating. a) Now we have pyenv/pipenv, should mention those, in preference to virtualenv (or conda-env) b) sudo considered harmful, and see the other answers why. So put the env-based answer first, and the sudo one last, with a big disclaimer. – Cantrell

You did a good job editing, can you just move 1) sudo pip install to the bottom, as it's now the least desirable approach, is a security hole, and is generally unnecessary. – Cantrell

sudo pip install -r requirements.txt is never right. The system's python environment belongs to the system, period. If you do install more python stuff into the system, do it with package manager only (e.g. sudo yum install, apt-get, etc...) since those repos should have safe and compatible versions of libraries avail. – Bodiless

I completely fail to see why there would not be a legitimate case to install something system-wide through a channel other than yum/apt-get, but we'll agree to disagree here. By that line of reasoning, there isn't a case for installing something by hand either - i.e. the old sudo make install? – Lazaretto

@TobiaTesan The old sudo make install, usually compiled + linked code, is not really analogous with a sudo pip install since installing to the system Python env can invalidate dependencies. Suppose there is a system service python-frobnicator, which has a dependency on froblib (this will also be in the package manager and pinned to a compatible version), and then you sudo pip install some other app or lib that has a dependency on "froblib > 1.2". Pip will happily "upgrade" the system version of froblib with a newer one, which may be incompatible/untested and break the system. – Bodiless

Note there can only be one version installed per-env, this is why Python has virtualenv. – Bodiless

@Bodiless I see what you mean now, and seems to me like a good point to make. Thanks! – Lazaretto

Answer 4

Just clarifying what worked for me after much pain in linux (ubuntu based) on permission denied errors, and leveraging from Bert's answer above, I now use ...

$ pip install --user <package-name>

or if running pip on a requirements file ...

$ pip install --user -r requirements.txt

and these work reliably for every pip install including creating virtual environments.

However, the cleanest solution in my further experience has been to install python-virtualenv and virtualenvwrapper with sudo apt-get install at the system level.

Then, inside virtual environments, use pip install without the --user flag AND without sudo. Much cleaner, safer, and easier overall.

Neva

I get a "Can not perform a '--user' install. User site-packages are not visible in this virtualenv." error when trying to use pip install --user -r requirements.txt – Kero

@AmirA.Shabani the answer has been edited since your question. It now says « inside virtual environments, use pip install without the --user flag AND without sudo » – Attrahent

Answer 5

User doesn't have write permission for some Python installation paths. You can give the permission by:

sudo chown -R $USER /absolute/path/to/directory

So you should give permission, then try to install it again, if you have new paths you should also give permission:

sudo chown -R $USER /usr/local/lib/python2.7/

Redhot

For python installed with brew, this is the right answer because brew maintains packages as the local user (no root). – Ananthous

chowning the /usr/local dir is not a good idea. It does not belong to the user. You should read about unix file structure. – Florencio

Stuff under /usr would typically be owned by root, these days. Recursively chowning there could majorly screw up your system. AVOID. – Bodiless

Answer 6

If you need permissions, you cannot use 'pip' with 'sudo'. You can do a trick, so that you can use 'sudo' and install package. Just place 'sudo python -m ...' in front of your pip command.

sudo python -m pip install --user -r package_name

Manon

Seems fine to me, but could you please add some explanation. – Kiyohara

Try --user before going the sudo route – Stlouis

Answer 7

It was worked for me by below commands

pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org PACKAGE_NAME --user

or

pip install --user <package name>

Example:

pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org matplotlib --user

Leinster

Answer 8

So, I got this same exact error for a completely different reason. Due to a totally separate, but known Homebrew + pip bug, I had followed this workaround listed on Google Cloud's help docs, where you create a .pydistutils.cfg file in your home directory. This file has special config that you're only supposed to use for your install of certain libraries. I should have removed that disutils.cfg file after installing the packages, but I forgot to do so. So the fix for me was actually just...

rm ~/.pydistutils.cfg.

And then everything worked as normal. Of course, if you have some config in that file for a real reason, then you won't want to just straight rm that file. But in case anyone else did that workaround, and forgot to remove that file, this did the trick for me!

Ventricose

Answer 9

It is due permission problem,

sudo chown -R $USER /path to your python installed directory

default it would be /usr/local/lib/python2.7/

or try,

pip install --user -r package_name

and then say, pip install -r requirements.txt this will install inside your env

dont say, sudo pip install -r requirements.txt this is will install into arbitrary python path.

Kazimir

As already noted in another older answer with the same advice, don't chown system files to belong to regular users. This is just the sudo pip install problem in spades. If you have to do one of those, sudo pip install is way better. – Feminine