How to make Python read from local dist-packages or site-packages instead of global dist-packages? ModuleNotFoundError: No module named 'azure.cosmos'

Question

First of all, this answer explains three python package locations nicely, I am giving a brief summary of them first, in my case for the azure-cosmos python package these are:

First, Global dist-packages, controlled with sudo apt-get (before installing a newer version with pip3 it shows ver 3.1.1):

$ sudo apt-get install python3-azure-cosmos
...
$ pip3 list -v | grep azure-cos
azure-cosmos 3.1.1 /usr/lib/python3/dist-packages

Second, Local dist-packages, installing with sudo pip3

$ sudo pip3 install -U azure-cosmos
...
$ pip3 list -v | grep azure-cos
azure-cosmos 4.2.0 /usr/local/lib/python3.8/dist-packages pip

Third, Local site-packages, (removed with sudo pip and then) installed with pip3 (without sudo)

$ sudo pip3 install -U azure-cosmos
...
$ pip3 list -v | grep azure-cos
azure-cosmos 4.2.0 /home/me/.local/lib/python3.8/site-packages pip

However, in all three cases my Python is looking ONLY in the (first) global dist-packages (that contains an older package version):

$ python3 -c "from azure.cosmos import CosmosClient"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'CosmosClient' from 'azure.cosmos' (/usr/lib/python3/dist-packages/azure/cosmos/__init__.py)

I do have $PATH and $PYTHONPATH set properly to include the local dist-packages and local site-packages locations, Python actually shows them in sys.path before global dist-packages:

$ python3 -c "import sys; print([p for p in sys.path])"
['', '/usr/local/lib/python3.8/dist-packages', '/home/me/.local/lib/python3.8/site-packages', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/lib/python3/dist-packages']

---

So why is Python taking the old version of azure-cosmos that came with the distro instead of getting a newer version that I have installed with pip? What am I missing here?

(I did try to update the python3-azure-cosmos package but 3.1.1 seems to be highest available via apt-get, and that's not the point of the question)

---

An easy solution is to remove the GLOBAL dist-packages version, one can simply run:

$ sudo apt-get remove python3-azure-cosmos

Then Python magically uses a version from either local dist-packages or site-packages from my home dir (package installed via sudo pip3 or pip3 respectively), because the package does not exist in global dist-packages.

But... my problem is that I cannot run sudo in this case. So I cannot remove the global package. Halp.

Nemato

Answer 1

Try to use the below command to Uninstall/Remove the packages without using sudo privileges rm -rf <Package_Directory>

Linux rm -rf is a powerful command deletes directory forcefully. It means a file or directory will be removed. Even though it has read-only permission.

Overtire

Without sudo I don't have permissions to remove from /ust/lib/python3/dist-packages – Nemato

Answer 2

A possible workaround (but not a solution) to this and a similar issue I had one day later is to just use a python virtual environment, e.g. using pipenv:

$ pip3 install --upgrade pipenv
$ pipenv install azure-cosmos
$ pipenv run python -c "from azure.cosmos import CosmosClient"

And we have empty output so there is no import error anymore, meaning that it works.

Nemato