How to install 3rd party module for postgres pl/python?

Question

I need to import a 3rd party module inside my pl/python function. It seems pl/python uses an internal python that does not have any 3rd party modules.

I get this kind of error:

ERROR:  PL/Python: PL/Python function "to_tsvector_luc" failed
DETAIL:  <type 'exceptions.ImportError'>: No module named lucene

********** Error **********

ERROR: PL/Python: PL/Python function "to_tsvector_luc" failed
SQL state: XX000
Detail: <type 'exceptions.ImportError'>: No module named lucene

How do I install the module into pl/python, so that I can import it from inside my stored procedure code?

Scouring

As you found out plpython3u uses the system Python and system site-packages not the local(user) packages. It makes sense as it running in a system service, the Postgres server. – Heilner

Answer 1

pl/python has access to the all the modules that the normal Python interpreter would have as long as they are in the $PYTHONPATH on the server (and the user that runs the postgres service). Does import lucene work if you run it in the Python interpreter on the server?

If your module is installed somewhere else (e.g. not dist-packages etc.), then you would need to edit your /etc/postgresql/9.1/main/environment (adjust to your PostgreSQL version) file on the server and add something like PYTHONPATH='<path to your module>'.

Pustule

/etc/postgresql/9.1/main/environment only appears to be a thing on Ubuntu, see here – Montgomery

And about plpython3u? (today using with pg13)... 1. Checking the path of module by print(a_module.__file__), is it? 2. the [$PYTHONPATH ](postgresql.org/docs/current/plpython-envar.html) seems same for Python3, is it? 3. But about many modules? And a module installed by pip but at /home/myUser ... Please expand answer about modern (2021) details – Schramm

Answer 2

Since modifying PYTHONPATH of postgres user will likely need a server restart, it's somewhat easier to add the path from within Python, via

from sys import path
path.append( '/path/to/your/module' )

Pointdevice

Not working... Need to add import moduleX clause? about path, is the folder where you see __init__.py , by print(a_module.__file__)? I installed chevron by pip and obtained a user-area path, /home/myUser/.local/lib/python3.8/site-packages/chevron/ , it is valid? No PostgreSQL server access-to-path problem? – Schramm

Answer 3

For me, it was about knowing which version Python Postgres was looking at and then installing to the /local/lib directory and NOT .local directory which is not recognised in Postgres.

In Postgres, created this function to identify the Python version it was using: .

CREATE OR REPLACE FUNCTION python_version()
    RETURNS pg_catalog.text AS $BODY$

    import sys
    plpy.info(sys.version)    
    return 'finish'
    $BODY$
LANGUAGE plpython3u VOLATILE SECURITY DEFINER

Execute function using the following:

select python_version()

In Python:

import sys
sys.version

Both the script and Python said: INFO: 3.7.3 (default, Aug 20 2019, 17:04:43)

pip3 install placed the library into this directory: /home/username/.local/lib/python3.7/site-packages

To make the package available to ALL users (including Postgres). I used umask:

sudo su
cd ~
umask 022
pip3 install <package name>

NOTE: The package installs in: /usr/local/lib# cd python3.7/dist-packages NOT in /usr/lib/python3/dist-packages#

Penrose

Yes, need ugly pip3 procedure with sudor and unmask 022. – Schramm

Answer 4

I think the 43 pl/python has access to the all the modules that the normal Python interpreter would have as long as they are in the Pythonpath on the server and user that is using PostgreSQL.

If your module is installed somewhere else (e.g. packages), then you would need to edit your /src/postgresql/9.1/main/environment (adjust to your PostgreSQL version) file on the server and add something like Pythonpath=>*'path to your module'**.

Martie

Answer 5

For Ubuntu 22.04:

• Install your module: sudo pip install foo.
• Find out where your python stores installed modules. For me it was /usr/local/lib/python3.10/dist-packages/ - where obviously python3.10 is your installed python version. The command pip3 show <your package name here> should help you out, if your path is different. E.g.: pip3 show numpy.
• Now, you need to change the permissions to your module files, because postgres is running as postgres user, but the files of your module will belong to root. You should do: sudo chmod -R 755 <path to your module>. E.g.: sudo chmod -R 755 /usr/local/lib/python3.10/dist-packages/test_module.

And now your module should be working.

Subconscious