How to run a script using pyproject.toml settings and poetry?

Question

I am using poetry to create .whl files. I have an FTP sever running on a remote host. I wrote a Python script (log_revision.py) which save in a database the git commit, few more parameters and in the end sends the .whl (that poetry created) to the remote server (each .whl in a different path in the server, the path is saved in the DB).

At the moment I run the script manually after each time I run the poetry build command. I know the pyproject.toml has the [tool.poetry.scripts] but I don't know how I can use it to run a Python script.

I tried:

[tool.poetry.scripts]
my-script = "my_package_name:log_revision.py

and then poetry run my-script but I always get an error:

AttributeError: module 'my_package_namen' has no attribute 'log_revision'

How can I run this command? As a short term option (without git and params) I tried to use poetry publish -r http://192.168.1.xxx/home/whl -u hello -p world but I get the following error:

[RuntimeError]                                 
Repository http://192.168.1.xxx/home/whl is not defined  

What am I doing wrong and how can I fix it?

Sanious

Answer 1

At the moment the [tool.poetry.scripts] sections is equivalent to setuptools console_scripts.

So the argument must be a valid module and method name. Let's imagine within your package my_package, you have log_revision.py, which has a method start(). Then you have to write:

[tool.poetry.scripts]
my-script = "my_package.log_revision:start"

Here's a complete example:

You should have this folder structure:

my_package
├── my_package
│   ├── __init__.py
│   └── log_revision.py
└── pyproject.toml

The complete content of a functional pyproject.toml is:

[tool.poetry]
name = "my_package"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.8"

[tool.poetry.scripts]
my-script = "my_package.log_revision:start"

[build-system]
requires = ["poetry_core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

and in my_package/log_revision.py:

def start():
    print("Hello")

After you have run poetry install once you should be able to do this:

$ poetry run my-script  
Hello

You cannot pass something to the start() method directly. Instead you can use command line arguments and parse them, e.g. with python's argparse.

Spallation

thx for the link, tried it out it is only work for me when setting my-script = log_revision:start ( could not enter the package name). in addition the log_revision.py should be in the same level folder as the pyproject.toml – Sanious

is there a way to send some data to the function log_revision:start(arg1, arg2)? – Sanious

helpper I had a hard time finding an example of how to pass arguments to the function, so I asked this question/answer – Alkanet

How do you set this up so you can run the command directly like you can with flask run my_app with setuptools has entry_points={"console_scripts": ["superset=superset.cli:superset"]} – Endure

Poetry 1.1.13 raises The Poetry configuration is invalid: - Additional properties are not allowed ('scritps' was unexpected) – Hendiadys

@Prometheus scritps looks like you've made a typo. – Copestone

How do you get the VSCode Python debugger working with this setup? – Rhinoceros

Answer 2

Although the previous answers are correct, they are a bit complicated. The simplest way to run a python script with poetry is as follows:

poetry run python myscript.py

If you are using a dev framework like streamlit you can use

poetry run streamlit run myapp.py

Basically anything you put after poetry run will execute from the poetry virtual environment.

Apron

Running a python script puts its folder at the beginning of the sys.path. If your script (or an imported module) imports http and there happens to be a an http.py in your folder, you will NOT get the python http module... – Grunenwald

This pattern is useful for shell scripts / Makefiles. For example if you want to add a make run target to your repo because your fingers are tired, this works perfectly. – Confetti

Answer 3

For future visitors, I think what OP is asking for (a post build hook?) isn't directly supported. But you might find satisfaction from using a tool I wrote called poethepoet which integrates with poetry to run arbitrary tasks defined in the pyproject.toml in terms of shell commands or by referencing python functions.

For example you could define something like the following in your pyproject.toml

[tool.poe.tasks.log_revision]
script = "my_package.log_revision:main" # where main is the name of the python function in the log_revision module
help   = "Register this revision in the catalog db"

[tool.poe.tasks.build]
cmd  = "poetry build"
help = "Build the project"

[tool.poe.tasks.shipit]
sequence = ["build", "log_revision"]
help     = "Build and deploy"

And then execute and of the tasks with the poe CLI like the following which will run the other two tasks in sequence, thus building your project and running the deployment script in one go!

poe shipit

By default tasks are executed inside the poetry managed virtualenv (like using poetry run) so the python script can use dev dependencies.

If you need to define CLI arguments or load values into a task from a dotenv file (such as credentials) then this is also supported.

---

Update: poetry plugin support

poethepoet can now support post build hooks when used as a poetry plugin. For example when using poetry >=1.2.0b1 you can configure the following to run your log_revision task automatically after poetry build is run:

[tool.poe.poetry_hooks]
post_build = "log-revision"

[tool.poe.tasks.log-revision]
script = "scripts:log_revision"

Guerdon

Excellent! Your tool is great. I experience though a small initialization lag when running a task (I guess due to some internal process sub-spawning). – Amaranth

Answer 4

Tinkering with such a problem for a couple of hours and found a solution

I had a task to start the django server via poetry script.

Here are the directories. manage.py is in test folder:

├── pyproject.toml
├── README.rst
├── runserver.py
├── test
│   ├── db.sqlite3
│   ├── manage.py
│   └── test
│       ├── asgi.py
│       ├── __init__.py
│       ├── __pycache__
│       │   ├── __init__.cpython-39.pyc
│       │   ├── settings.cpython-39.pyc
│       │   ├── urls.cpython-39.pyc
│       │   └── wsgi.cpython-39.pyc
│       ├── settings.py
│       ├── urls.py
│       └── wsgi.py
├── tests
│   ├── __init__.py
│   └── test_tmp.py
└── tmp
    └── __init__.py

I had to create a file runserver.py:

 import subprocess                                                               
                                                                                   
  def djtest():                                                                   
      cmd =['python', 'test/manage.py', 'runserver']                                                                 
      subprocess.run(cmd)                                                    

then write the script itself pyproject.toml:

[tool.poetry.scripts]                                                           
dj = "runserver:djtest"  

and still make changes to pyproject.toml:

[tool.poetry.scripts]                                                           
dj = "runserver:djtest"

only then command poetry run dj started working

Hap

Do you know if calling python via subproc invoked your poetry venv or uses the system? – Pavilion

To answer my own comment, yes it picks up the poetry venv – Pavilion

only then "dj" starts working – Dyewood

what are the differences between the last to code blocks? – Messidor

Answer 5

I know the question is about poetry, but you can use the more general:

[project.scripts]
script_name = "your_package.a_file:a_method"

That way also users that do not have poetry can a easy to access CLI script.

Sixth