Launch an IPython shell on exception

Question

Is there a way to launch an IPython shell or prompt when my program runs a line that raises an exception?

I'm mostly interested in the context, variables, in the scope (and subscopes) where the exception was raised. Something like Visual Studio's debugging, when an exception is thrown but not caught by anyone, Visual Studio will halt and give me the call stack and the variables present at every level.

Do you think there's a way to get something similar using IPython?

EDIT: The -pdb option when launching IPython doesn't seem do what I want (or maybe I don't know how to use it properly, which is entirely possible). I run the following script :

def func():
    z = 2
    g = 'b'
    raise NameError("This error will not be caught, but IPython still"
                    "won't summon pdb, and I won't be able to consult"
                    "the z or g variables.")

x = 1
y = 'a'

func()

Using the command :

ipython -pdb exceptionTest.py

Which stops execution when the error is raised, but brings me an IPython prompt where I have access to the global variables of the script, but not the local variables of function func. pdb is only invoked when I directly type a command in ipython that causes an error, i.e. raise NameError("This, sent from the IPython prompt, will trigger pdb.").

I don't necessarily need to use pdb, I'd just like to have access to the variables inside func.

EDIT 2: It has been a while, IPython's -pdb option is now working just as I want it to. That means when I raise an exception I can go back in the scope of func and read its variables z and g without any problem. Even without setting the -pdb option, one can run IPython in interactive mode then call the magic function %debug after the program has exit with error -- that will also drop you into an interactive ipdb prompt with all scopes accessibles.

Variegate

When doing this kind of hacks, keep in mind that your script might hang whatever calls it if it's a part of another program. My suggestion is to check if sys.stdin.isatty() before doing that. – Sephira

Answer 1

Update for IPython v0.13:

import sys
from IPython.core import ultratb
sys.excepthook = ultratb.FormattedTB(mode='Verbose',
     color_scheme='Linux', call_pdb=1)

Etana

Should I put it on top of the driver file? I did but nothing happen. Please note that I am using ipdb – Bruch

How to enable from CLI? – Badminton

Answer 2

Doing:

ipython --pdb -c "%run exceptionTest.py"

kicks off the script after IPython initialises and you get dropped into the normal IPython+pdb environment.

Sports

Also, you can open python first, then use run -d <path to script>, and you will be dropped into IPython+pdb. – Nucleoprotein

This does not allow me to restart the code using restartcommand – Bruch

Answer 3

You can try this:

from ipdb import launch_ipdb_on_exception

def main():
    with launch_ipdb_on_exception():
        # The rest of the code goes here.
        [...]

Indorse

I think this is the 'modern' way to do it. – Stilbestrol

For the with launch_ipdb_on_exception(): line I am getting following lint message: Context manager 'generator' doesn't implement __enter__ and __exit__. (not-context-manager). How can I prevent this ? – Bruch

Answer 4

ipdb integrates IPython features into pdb. I use the following code to throw my apps into the IPython debugger after an unhanded exception.

import sys, ipdb, traceback

def info(type, value, tb):
    traceback.print_exception(type, value, tb)
    ipdb.pm()

sys.excepthook = info

Platon

ipdb also has a handy context manager: launch_ipdb_on_exception() – Mease

Answer 5

@snapshoe's answer does not work on newer versions of IPython.

This does however:

import sys 
from IPython import embed

def excepthook(type, value, traceback):
    embed()

sys.excepthook = excepthook

Dylan

This is the one I've used for starting the shell at some point in a script – Tuberculin

Answer 6

You can do something like the following:

import sys
from IPython.Shell import IPShellEmbed
ipshell = IPShellEmbed()

def excepthook(type, value, traceback):
    ipshell()

sys.excepthook = excepthook

See sys.excepthook and Embedding IPython.

Cotidal

I love this idea, but I read here that ipython does not use sys.excepthook. – Variegate

Try adding those lines to the top of your script, then launch it outside of ipython, and verify that it does drop you into an ipython session. If you started the script with ipython <script>, then you're already in ipython and don't need to launch ipython. IPython does use sys.excepthook-- ipython itself replaces the hook with it's own handler. Within ipython, run import sys; sys.excepthook to verify this. – Cotidal

Ok, that worked and catched the exception in an ipython shell, but I still didn't have access to the variables declared in the scope of the function (z,g). – Variegate

OK, at the time of calling ipshell, I am no longer in the scope of my function, so it normal that those variables are not accessible, however is there a way, using pdb or something else, to get back to the scope of the function that launched this exception? – Variegate

@Variegate ipshell can take the argument stack_depth. If you want your IPython shell to load the variables from one level up in the call stack, you can call ipshell(stack_depth=2) (level 1 is the default, which is the context where ipshell() is called). – Dominick

Answer 7

@Adam's works like a charm except that IPython loads a bit slowly(800ms on my machine). Here I have a trick to make the load lazy.

class ExceptionHook:
    instance = None

    def __call__(self, *args, **kwargs):
        if self.instance is None:
            from IPython.core import ultratb
            self.instance = ultratb.FormattedTB(mode='Verbose',
                 color_scheme='Linux', call_pdb=1)
        return self.instance(*args, **kwargs)
sys.excepthook = ExceptionHook()

Now we don't need to wait at the very beginning. Only when the program crashes will cause IPython to be imported.

Jorgensen

Answer 8

This man page says iPython has --[no]pdb option to be passed at command line to start iPython for uncaught exceptions. Are you looking for more?

EDIT: python -m pdb pythonscript.py can launch pdb. Not sure about similar thing with iPython though. If you are looking for the stack trace and general post-mortem of the abnormal exit of program, this should work.

Occlude

-1 This is for uncaught exceptions when something is already running within ipython, not for starting ipython when it isn't running yet. – Cotidal

Answer 9

If you want to both get the traceback and open a IPython shell with the environment at the point of the exception:

def exceptHook(*args):
    '''A routine to be called when an exception occurs. It prints the traceback
    with fancy formatting and then calls an IPython shell with the environment
    of the exception location.
    '''
    from IPython.core import ultratb
    ultratb.FormattedTB(call_pdb=False,color_scheme='LightBG')(*args)
    from IPython.terminal.embed import InteractiveShellEmbed
    import inspect
    frame = inspect.getinnerframes(args[2])[-1][0]
    msg   = 'Entering IPython console at {0.f_code.co_filename} at line {0.f_lineno}'.format(frame)
    savehook = sys.excepthook # save the exception hook
    InteractiveShellEmbed()(msg,local_ns=frame.f_locals,global_ns=frame.f_globals)
    sys.excepthook = savehook # reset IPython's change to the exception hook

import sys
sys.excepthook = exceptHook

Note that it is necessary to pull than namespace information from the last frame referenced by the traceback (arg[2])

(12/23) The above code needs to change to match the latest versions of IPython. Below is what I am using now inside my exceptHook function. I have tried it on IPython 8.x and 7.x and it might work with even older versions, but has not been tested.

try: 
    from IPython.terminal.embed import InteractiveShellEmbed
    import IPython.core
    if sys.platform.startswith('win'):
        IPython.core.ultratb.FormattedTB(
           call_pdb=False,color_scheme='NoColor')(*args)
    else:
        IPython.core.ultratb.FormattedTB(
           call_pdb=False,color_scheme='LightBG')(*args)
    from IPython.core import getipython
    if getipython.get_ipython() is None:
        ipshell = InteractiveShellEmbed.instance()
    else:
        ipshell = InteractiveShellEmbed()
except ImportError:
    print ('IPython not found or really old')
    return

import inspect
frame = inspect.getinnerframes(args[2])[-1][0]
msg   = 'Entering IPython console at {0.f_code.co_filename} at line {0.f_lineno}\n'.format(frame)
savehook = sys.excepthook # save the exception hook
try:
    ipshell(msg,local_ns=frame.f_locals,global_ns=frame.f_globals) # newest (IPython >= 8)
except DeprecationWarning: # IPython <=7
    try: # IPython >=5
        class c(object): pass
        pseudomod = c() # create something that acts like a module
        pseudomod.__dict__ = frame.f_locals
        InteractiveShellEmbed(banner1=msg)(module=pseudomod,global_ns=frame.f_globals)
    except: # 'IPython <5
        InteractiveShellEmbed(banner1=msg)(local_ns=frame.f_locals,global_ns=frame.f_globals)
sys.excepthook = savehook # reset IPython's change to the exception hook

Ventriloquize

If anyone is still using IPython 0.12 (EPD), the import needs to be from IPython.frontend.terminal.embed import InteractiveShellEmbed – Ventriloquize

This is the best answer. It's the only one that lets you write Python code within the (approximate) context of the exception, blurring the line between debugging and development. – Hylan

Answer 10

Do you actually want to open a pdb session at every exception point? (as I think a pdb session opened from ipython is the same as the one open in the normal shell). If that's the case, here's the trick: http://code.activestate.com/recipes/65287-automatically-start-the-debugger-on-an-exception/

Capsicum