How to import an existing requirements.txt into a Poetry project?

Question

I am trying out Poetry in an existing project. It used pyenv and virtual env originally so I have a requirements.txt file with the project's dependencies.

I want to import the requirements.txt file using Poetry, so that I can load the dependencies for the first time. I've looked through poetry's documentation, but I haven't found a way to do this. Is there a way to do it?

I know that I can add all packages manually, but I was hoping for a more automated process, because there are a lot of packages.

Infrequent

Did you try pip freeze > requirements.txt on command line? – Copula

Yes, I have the requirements.txt file. I would like to import it into Poetry without having to type in the packages manually. – Infrequent

did you generate it by freezing the complete environment, or did you maintain it by hand? note that there is a big difference between abstract and concrete project requirements. poetry will generate the latter for automatically as a poetry.lock file, but the abstract requirement list both needs to and should be maintained by hand. And it most definitely can't be extracted from the result of a pip freeze. – Clifford

It is maintained by hand. It is an abstract requirement, since it only lists the higher level packages. – Infrequent

that's good. projects will seldom have more than a handfull of abstract dependencies, so I kind of assumed it might be a concrete list. But yeah, poetry doesn't have a command to import requirements.txt. You can use dephell, but I don't know how good or reliable that is. Honestly, I'd always do dependency porting by hand, since it's one of the parts of an app that can lead to serious problems and technical debt if it's not cared for as good as possible. – Clifford

Answer 1

poetry doesn't support this directly. But if you have a handmade list of required packages (at best without any version numbers), that only contain the main dependencies and not the dependencies of a dependency you could do this:

$ cat requirements.txt | xargs poetry add

Chobot

For those reading this beware if your requirements include alpha-versioned packages: github.com/python-poetry/poetry/issues/4653 – Adroit

If you do have version numbers you could modify this with cat requirements.txt | xargs -I % sh -c 'poetry add "%"' – Egomania

You can avoid the unnecessary cat with xargs poetry add < requirements.txt – Jenna

This is nice but has 2 problems: 1) It doesn't stop on first error and keeps running poetry add 2) It won't work if requirements.txt is using some exotic encoding. My answer addresses both problems and provides extra features. – Ennui

@Mateen Ulhaq your photography skills though <3 – Vigue

How to do this in \windows ? – Underbrush

even chatGPT couldnt get this – Immensity

It's really an issue that Poetry team should deal with. – Obsolesce

Please note that a poetry init (for existing projects) or poetry new is necessary to do this. I like @EllisPercival's solution without cat. – Congressman

Answer 2

poetry add $(cat requirements.txt)

Greenish

@Ruli this is a complete answer. It is clear and needs no further explanation, and it just works. This is the best answer here. – Semipalmate

also poetry add $(cat requirements.txt) works better for those who hate backticks :-D – Anteversion

This also works with versions in requirements.txt – Patellate

To ignore comments, poetry add $(sed 's/#.*//' requirements.txt) works for me. – Electrophilic

Answer 3

I don't have enough reputation to comment but an enhancement to @Liang's answer is to omit the echo and call poetry itself.

cat requirements.txt | grep -E '^[^# ]' | cut -d= -f1 | xargs -n 1 poetry add

In my case, this successfully added packages to the pyproject.toml file.

For reference this is a snippet of my requirements.txt file:

pytz==2020.1  # https://github.com/stub42/pytz
python-slugify==4.0.1  # https://github.com/un33k/python-slugify
Pillow==7.2.0  # https://github.com/python-pillow/Pillow

and when calling cat requirements.txt | grep -E '^[^# ]' | cut -d= -f1 (note the omission of xargs -n 1 poetry add for demonstration) it will output the following:

pytz
python-slugify
Pillow
# NOTE: this will install the latest package - you may or may not want this.

Adding dev dependencies is as simple as adding the -D or --dev argument.

# dev dependancies example
cat requirements-dev.txt | grep -E '^[^# ]' | cut -d= -f1 | xargs -n 1 poetry add -D

Lastly, if your dev requirements install from a parent requirements file, for example:

-r base.txt

package1
package2

Then this will generate errors when poetry runs, however, it will continue past the -r base.txt line and install the packages as expected.

Tested on Linux manjaro with poetry installed as instructed here.

Rodman

Answer 4

A one-liner for Windows PowerShell users:

    @(cat requirements.txt) | %{&poetry add $_}

For more about piping arguments with PowerShell see this useful answer.

Danieledaniell

Answer 5

I made a tool poetry-add-requirements.txt just for this. Code

Install it with pipx install poetry-add-requirements.txt,

then run poeareq.

Usage

Run poetry-add-requirements.txt, optionally specify your requirements.txt files and --dev for dev dependencies.

poeareq is provided is an alias to poetry-add-requirements.txt.

$ poeareq --help

usage: poeareq [-h] [-D] [requirements.txt files ...]

Add dependencies specified in requirements.txt to your Poetry project

positional arguments:
  requirements.txt file(s)
                        Path(s) to your requirements.txt file(s) (default: requirements.txt)

options:
  -h, --help            show this help message and exit
  -D, --dev             Add to development dependencies (default: False)

Features

• Auto detect charset of requirements.txt file(s) and feed normalized dependency specs to poetry.
• Stop on first poetry add error.

Ennui

Answer 6

The best method I've found is this one (unix shell command):

for item in $(cat requirements.txt); do poetry add "${item}"; done

Encrinite

I think the first $ is stray character and is not required – Counterintelligence

You might find man xargs interesting ... ;). This is basically the same as accepted answer. Fun thing about unix, lots of ways to do things! – Sporogonium

foreach ($item in $(wsl cat requirements.txt)){poetry add $item} here is a powershell version of the answer. – Mut

@Counterintelligence Often, the first $ character is used to show this line should be entered. When there are multiple lines, the lines with $ should be entered, and the lines without $ are output of the command. – Helotism

Answer 7

Just use the plain requirements.txt and filter out version numbers with awk:

awk -F '==' '{print $1}' requirements.txt | xargs -n1 poetry add

-F specifies a filter or split point. $1 is the first argument in the split. The input file comes as last argument. Afterwards you can pipe it to poetry add using xargs -n 1 to call poetry add with each line consecutively and not with a space separated string at once. If you want to consume all entries at once just ommit -n 1. Also make sure that a poetry environment is already present.

To just consume the requirements.txt omit the filter and use

awk '{print $1}' requirements.txt | xargs -n1 poetry add

But other tools like cat are fine for that case as well.

Steiermark

Answer 8

One liner:

cat requirements.txt | grep -E '^[^# ]' | cut -d= -f1 | xargs -n 1 poetry add

Goodell

If you have a line Pillow>=9.5.0, an error will be shown: Could not parse version constraint: > – Rosauraroscius

Answer 9

Here's one that works if you have # comments (at the start of a line or at the end of a line) in your requirements file:

poetry add $(sed -e 's/#.*//' -e '/^$/ d' < requirements.txt)

https://www.grymoire.com/Unix/Sed.html#uh-30

Elastomer

You don't need < to feed sed: sed -e 's/#.*//' -e '/^$/ d' requirements.txt is enough. – Pileous

What is the point of this when you can just do pip install -r requirements.txt? – Cleaner

@Cleaner OP wants to manage their dependencies with poetry. – Elastomer

Answer 10

Very short for Linux or MacOS:

poetry add $(grep -v '^#' requirements.txt)

If you want to use data from multiple files, you can use the following example.

poetry add $(grep -vh '^#' path/to/files/*.txt)

Bullpen

Answer 11

For Windows' users

In Windows, the xargs command, which is commonly used in Unix-like systems, is not a standard command. However, you can use an alternative approach to achieve a similar result using the PowerShell commandlet.

Use this Get-Content requirements.txt | ForEach-Object { poetry add $_ }

This command reads the content of the requirements.txt file using Get-Content and then passes each line to poetry add using ForEach-Object. Each line from the requirements.txt file is passed as an argument to poetry add, and the dependencies are added to your project using Poetry.

You must be in the root directory of your project

Luann

Answer 12

For Powershell:

$reqs = @(cat requirements.txt)
for($i = 0; $i -lt $reqs.length; $i++){poetry add $reqs[i]}

Note this won't ignore comments or anything else in the requirements file. This is strictly taking it as raw text so it expects every line to be a package.

Vardon

I had to use $i in $args[i] so its for($i = 0; $i -lt $reqs.length; $i++){poetry add $reqs[$i]} – Decoteau

Answer 13

I found none of these answers sufficed so I created one of my own:

https://github.com/src-r-r/python-stanza

It's a new baby, so contributions welcome, but so far it's very cookiecutter-friendly:

• automatically detects a setup.py and fetches project info
• allows multiple requirements.txt files to be specified for either dev dependencies or normal dependencies
• allows the name and version to be overwritten.
• also adds referenced requirements (e.g. -r ./special-requirements.txt) if it's included in a requirements file.

Reachmedown

Answer 14

For Windows users using the Windows Command Prompt (and not Powershell), this'll work

FOR /F "usebackq delims=" %G IN (requirements.txt) DO poetry add --lock %G

(Update: if poetry complains that "Poetry could not find a pyproject.toml file in or its parents", just execute poetry init first).

Julius

Answer 15

Had this issue when moving from a requirement.txt file to using Poetry.

If you want to run the command in windows using cmd you can run it as a .bat file:

for /f "tokens=*" %%i in (requirements.txt) do (
    poetry add %%i
)

Esquire

Answer 16

If anyone is wondering what should be the code for executing the same in Windows CMD is as below:

for /f %i in (requirements.txt) do (poetry add %i)

Nobility

Answer 17

most of the above will fail with the versions specified, this worked for me though.

usage reference

cat requirements.txt | xargs -I % sh -c 'poetry add "%"'

Drawstring

Answer 18

for PowerShell on Windows

PS > gc .\requirements.txt | ? { $_ -ne "" } | % { poetry add $_ }

Updating dependencies
Resolving dependencies... (1.5s)

Package operations: 9 installs, 0 updates, 0 removals

Remmer