ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device

Question

I am facing an error to install the packages on aws ec2 instance with Ubuntu 18 using the following command -

pip install -e .

The error is -

ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device

What did I check?

1. RAM using free -h command.
2. Disk utilization using sudo ncdu -x command.
3. Since pip tries to download to the default location given by TMPDIR variable, I also removed files from that location.
4. I removed contents from .cache directory.
5. I removed contents from /tmp directory.

Still I am facing this issue.

Coxswain

Answer 1

The answer provided at https://github.com/pypa/pip/issues/5816#issuecomment-425410189, states that

pip downloads files to temporary directory, environment variable TMPDIR specifies that directory, also pip puts files into cache thus --cache-dir specification, --no-cache-dir should work too. --build specifies directory where wheel will be built, so its specification is also useful.

For my user, I made a custom directory named codebase/pip_cache/ in my home directory.

First I tried --no-cache-dir using the following command -

TMPDIR=/home/deepakahire/codebase/pip_cache/ pip install -e . --no-cache-dir. This didn't work.

Finally, I specified the --cache-dir as well, and used the following command to install the package -

TMPDIR=/home/deepakahire/codebase/pip_cache/ pip install --cache-dir=/home/deepakahire/codebase/pip_cache/ -e . This worked for me.

Caveat- Blindly deleting everything from /tmp directory would delete your tmux sessions, but will still keep the services/instances alive on the same ports, which were switched on in any tmux session previously.

Coxswain

Thanks, I think this is more in alignment with newer versions of PIP, as it complained --build wasn't recognized. But I also had to create a swap partition for on my EC2 because it kept running out of memory just from downloading the file. – Comic

After installing like this, my venv doesnt recognize the installed packages, any idea what could go wrong? – Bodiless

I was able to fix this issue on a t2.micro AWS Linux machine by running a similar command: python3.11 -m pip install -r requirements.txt --cache-dir=/home/ec2-user/pip_cache/ – Caracas

Answer 2

I was running into this error on an AWS EC2 instance inside a python environment.

I noticed that when I ran watch -n 0.1 df -h /tmp in a separate terminal, and then I ran the pip install torch command in the main terminal, that the Use% went to 100%, and that is when I got the OSError(28, 'No space left on device') error.

This happens when the /tmp directory does not have enough space for downloading the archives and unpacking them, so you need to set the /tmp directory to a directory with sufficient space.

Running df -h shows you which mounted drives have enough space, so you can choose one of those and create a tmp_pip directory there, and then run commands like this:

mkdir  /home/ec2-user/pip_cache
export TMPDIR=/home/ec2-user/pip_cache
pip install torch

That ended up working for me, hope that helps!

Commeasure

Rather than messing with important environment variables, it's best to do this per-command like TMPDIR=/home/ec2-user/pip_cache pip install torch as was suggested in the other answer. – Risarise