How to run pytest with an python debugger with tab completion?

Question

Background

I have been debugging my python scripts for ~2 years with plain from IPython import embed; embed(), and it has been working really fine. I just place the command on the line I want to examine, and when running the script I will have full IPython shell with capability for examining variables, defining functions, etc. On top of that, IPython shell comes with variable name tab completion.

Now, instead of always defining the "line of pause" with from IPython import embed; embed(), I would like my python scripts pause the execution while running tests, when they encounter an exception.

Question

How do you run pytest in such way, that

1. When an exception is raised, it launches python debugger
2. This debugger has tab completion for variable names?

Setup

• Windows 10
• Python 3.6

Simple failing test

# test_somemodule.py
def test_me(some_variable):
    x = 1 + some_variable
    return x


test_me('I am a string')

(No, you do not write tests for pytest like this, but for this exemplary purpose this is just fine.)

Solutions that do not work

1. pytest --pdb (with no pdbpp installed)

This opens the basic pdb shell when it runs to the error. But it has no tab completion.

-> x = 1 + some_variable
(Pdb) so[<tab_here_produces_tab>]

2. pytest --pdb (with pdbpp installed)

This opens the basic pdbpp shell when it runs to the error. But no tab completion.

-> x = 1 + some_variable
(Pdb++) so[<tab_here_produces_tab>]

3. pytest --pdb --pdbcls=IPython.terminal.debugger:Pdb

This opens the basic ipdb shell when it runs to the error. But no tab completion.

    103
    104 def test_me(some_variable):
--> 105     x = 1 + some_variable
    106     return x
    107

ipdb> so[<tab_here_produces_tab>]

4. fancycompleter.interact()

Tried fancycompleter.interact() as suggested here, no luck (with pdb, ipdb and pdbpp, fancycompleter v.0.8 and even this patched version.).

5. pytest --pdb -s

As Sergey Voronezhskiy commented, there is -s flag available for pytest. However, the tab completion works only partially: If there are multiple options for the same starting character(s), it will print out list of possible variables. This lacks the ability of quickly selecting one of the matching variables (which is available in IPython shell, for example):

Is there a way to make the variable name tab completion work? I would prefer ipdb or pdbpp over the vanilla pdb, but even a working solution with pdb is just fine.

Antimacassar

Found some related discussion at github.com/pytest-dev/pytest/issues/4008 – Antimacassar

looks like you forgot -s option – Immovable

Thank you! Did not notice that flag. It's nice solution since it does not add an extra step, but there is the downside that it really does not allow tab completion if there are multiple variable names starting with the same letter(s). It just prints the available options. I will add an example picture. – Antimacassar

Have you tried pdbpp recently with latest version of pytest. Tab completion works fine on my machine with pdbpp on pytest. – Dyche

Yeah I tried now with pdbpp 0.10.2 and pytest 5.4.3 and there is no tab completion. Maybe some other package missing, or it is an issue of Windows / Powershell. – Antimacassar

I had the misfortune of using windows and pdbpp doesn't autocomplete there. I haven't tried other solutions mentioned here. – Dyche

Please see if this helps; basically you pip install pytest-pudb and then pytest ... --pubd – Gagman

Answer 1

^{This is not the optimal solution, but it's better than nothing. Maybe somebody comes with a better answer. Here is what I did}

1. alias emb for starting IPython.embed()

1.1. Steps

• Made ~/.pdbrc file (‪C:\Users\<USER>\.pdbrc) with following contents

alias emb from IPython import embed; embed()

1.2. Short explanation

• When pytest is ran with --pdb flag, it starts standard library pdb, or if installed, pdbpp.
• The standard library pdb documentation says:

If a file .pdbrc exists in the user’s home directory or in the current directory, it is read in and executed as if it had been typed at the debugger prompt. This is particularly useful for aliases. If both files exist, the one in the home directory is read first and aliases defined there can be overridden by the local file.

2. Reminder text for alias (optional)

2.1. Steps

• Installed pdbpp (pip install pdbpp)
• Made ~/.pdbrc.py¹ file (‪C:\Users\<USER>\.pdbrc.py) with following contents

import pdb


class Config(pdb.DefaultConfig):
    def setup(self, pdb):
        print('Use "emb" to enter IPython shell')

2.2. Short explanation

The pdbpp documentation says that if you create a ~/.pdbrc.py with a setup() method, it will be called when you enter the debugger. Therefore, it is possible to create a script that is called every time you enter pdbpp debugging shell.

^{1Note: This one has .py extension, while the first one, associated with the builtin pdb, does not.}

3. Usage Example

Run pytest --pdb (same .py file as in the question). As you can see from the figure, tab completion for the variable some_variable works.

Antimacassar

Better yet, if you like colour: alias emb from IPython import embed; embed(colors="neutral") – Starfish