Breakpoint-induced interactive debugging of Python with IPython

Question

Say I have an IPython session, from which I call some script:

> run my_script.py

Is there a way to induce a breakpoint in my_script.py from which I can inspect my workspace from IPython?

I remember reading that in previous versions of IPython one could do:

from IPython.Debugger import Tracer;     

def my_function():
    x = 5
    Tracer()
    print 5;

but the submodule Debugger does not seem to be available anymore.

Assuming that I have an IPython session open already: how can I stop my program a location of my choice and inspect my workspace with IPython?

In general, I would prefer solutions that do not require me to pre-specify line numbers, since I would like to possibly have more than one such call to Tracer() above and not have to keep track of the line numbers where they are.

Apical

Out of curiosity, why would you ever want more than one tracer at once? At that point it seems like you should be doing logging, not debugging. – Stadler

@Wilduck: I would like to inspect my workspace under certain circumstances, and those circumstances may occur in different locations in my code. – Apical

Okay, maybe I phrased my question poorly. I understand why you would want to look at different parts of your code. Why would you want to have those tracers in your code at the same time? If you're not using the debugger to permanently fix a bug (i.e. one bug at a time) wouldn't it be better/easier to set up logging to automatically collect information about how your program is running in various states? I'm asking because it seems that you're trying to use debugging tools for something logging takes care of. – Stadler

Thanks @Wilduck. I am not really interested in logging. I would like to mathematically and interactively examine the value of several variables when certain conditions are met. I could certainly dump the variables to disk and inspect them later, but that would not allow me to resume execution, which sometimes I want. This type of workflow is common and powerful in scientific computing (e.g. MATLAB supports this through the use of keyboard statements) and it allows one to inspect and visualize data interactively without fully disrupting or having to reinitialize a program. – Apical

I see, that does make sense. I'll leave my answer up, since it might be useful to someone stumbling on this question later, but I'm afraid I don't have any further suggestions for you personally. Good luck. – Stadler

Answer 1

The Tracer() still exists in ipython in a different module. You can do the following:

from IPython.core.debugger import Tracer

def my_function():
    x = 5
    Tracer()()
    print 5

Note the additional call parentheses around Tracer

edit: For IPython 6 onwards Tracer is deprecated so you should use set_trace() instead:

from IPython.core.debugger import set_trace

def my_function():
    x = 5
    set_trace()
    print 5

Toneytong

@Arnon how to use set_trace(); which set_trace? Plain old ipdb.set_trace() produces this problem for me, which advises to use Tracer()()! Oh, maybe the Pdb.set_trace() seen here or here ; the documentation linked describes IPython.core.debugger.set_trace, but I don't have that func. – Dreeda

The documentation linked is for IPython 6 (which only supports Python 3), and my suspicion is that you are using an older version. IPython.core.debugger.Tracer still exists and works in IPython 6, even if deprecated. I guess you can continue using it for some time until it is actually removed from the codebase. – Toneytong

Haha, that's true, I'm using Tracer nonetheless, hopefully someone can describe the recommended alternative? – Dreeda

Man, I wish IPython would stop moving the goalposts every couple of years – Loess

This answer is broken for the current version of python. – Malisamalison

Answer 2

With Python 3 (v3.7+), there's the new breakpoint() function. You can modify it's behaviour so it'll call ipython's debugger for you.

Basically you can set an environment variable that points to a debugger function. (If you don't set the variable, breakpoint() defaults to calling pdb.)

To set breakpoint() to call ipython's debugger, set the environment variable (in your shell) like so:

# for bash/zsh users
export PYTHONBREAKPOINT='IPython.core.debugger.set_trace'
# powershell users
$env:PYTHONBREAKPOINT='IPython.core.debugger.set_trace'

(Note, obviously if you want to permanently set the environment variable, you'll need to modify your shell profile or system preferences.)

You can write:

def my_function():
    x = 5
    breakpoint()
    print(5)

And it'll break into ipython's debugger for you. I think it's handier than having to import from IPython.core.debugger import set_trace and call set_trace().

Eleonoreleonora

adding ipython==8.8.0 in my requirements.txt lets me use ipython in pdb – Heydon

For me, on IPython v8.19.0, setting this evironment variable to IPython.**core**.debugger.set_trace does not provide autocompletion when entering the PDB. But, setting it to Ipython.**terminal**.debugger.set_trace will enable autocompletion ! In short, use : export PYTHONBREAKPOINT='IPython.terminal.debugger.set_trace' – Unsought

Answer 3

You can run it and set a breakpoint at a given line with:

run -d -b12 myscript

Where -b12 sets a breakpoint at line 12. When you enter this line, you'll immediately drop into pdb, and you'll need to enter c to execute up to that breakpoint.

Stadler

Thanks @Wildluck. This is definitely useful, although I am interested in something that does not require specifying line numbers. – Apical

Answer 4

This is the version using the set_trace() method instead of the deprecated Tracer() one.

from IPython.core.debugger import Pdb

def my_function():
    x = 5
    Pdb().set_trace()
    print 5

Worthington

I wanted to add this answer as a comment to the accepted answer above, but I don't have enough reputation to comment, so I hope if this answer helps anyone – Worthington

Alternatively, you can directly use IPython.core.debugger.set_trace() without needing Pdb. – Pontianak

Answer 5

Inside the IPython shell, you can do

from IPython.core.debugger import Pdb
pdb = Pdb()
pdb.runcall(my_function)

for example, or do the normal pdb.set_trace() inside your function.

Mattos

Thanks, but it's not clear how the first solution would allow me to specify possible locations of my choice for breakpoints. As for the second one, how would I actually use that option? Finally, from what I understand, pdb does not offer the same features that IPython does for introspection, etc. Is that correct? – Apical

You can use the b pdb command to define the breakpoints once you run the debugger. – Carinacarinate

Answer 6

I have always had the same question and the best workaround I have found which is pretty hackey is to add a line that will break my code, like so:

...
a = 1+2
STOP
...

Then when I run that code it will break, and I can do %debug to go there and inspect. You can also turn on %pdb to always go to point where your code breaks but this can be bothersome if you don't want to inspect everywhere and everytime your code breaks. I would love a more elegant solution.

Liebig

Answer 7

if you are using pytest then

pip install ipython
pip install ipdb
# run tests with options
pytest --pdb --pdbcls=IPython.terminal.debugger:TerminalPdb

# or set it in pytest.ini
[pytest]
addopts = --pdbcls=IPython.terminal.debugger:TerminalPdb

Hoar

Thank you! Do you know how to enable syntax highlighting? – Pillbox

Answer 8

I see a lot of options here, but maybe not the following simple option. Fire up ipython in the directory where my_script.py is. Turn the debugger on if you want the code to go into debug mode when it fails. Type %pdb.

In [1]: %pdb
Automatic pdb calling has been turned ON

Next type

In [2]: %run -d ./my_script.py
*** Blank or comment
*** Blank or comment
NOTE: Enter 'c' at the ipdb>  prompt to continue execution.
> c:\users\c81196\lgd\mortgages-1\nmb\lgd\run_lgd.py(2)<module>()
      1 # system imports
----> 2 from os.path import join

Now you can set a breakpoint where ever you want it. Type b 100 to have a breakpoint at line 100, or b whatever.py:102 to have a breakpoint at line 102 in whatever.py. For instance:

ipdb> b 100

Then continue to run, or continue.

ipdb> c

Once the code fails, or reaches the breakpoint you can start using the full power of the python debugger pdb.

Note that pdb also allows the setting of a breakpoint at a function.

b(reak) [([filename:]lineno | function) [, condition]]

So you do not necessarily need to use line numbers.

Terrellterrena