Update a function during debugging (pdb or ipdb)

Question

Imagine I am debugging the following script:

import ipdb

def slow_function(something):
  # I'm a very slow function
  return something_else

def fast_function(something_else):
  # There's a bug here
  return final_output

something = 1
something_else = slow_function(something)
ipdb.set_trace()
final_output = fast_function(something_else)
ipdb.set_trace()

When the ipdb.set_trace() line is met the debugger shell is prompted and I can now execute the final_output = fast_function(something_else) statement to check whether fast_function is behaving as intended. I see there's a bug so I go into the source code and I fix it. Now I want to see whether the fix is correct but I don't want to run the script a second time (because it's slow), nor I want to save something_else on disk (because, maybe, it's very large).

Is there a way to update fast_function() in the debugger shell so that the new source code is used?

Urethrectomy

Note that, depending on what you're doing, the answers below may or may not work in all situations. The reason is that if there are other references to fast_function, they won't be updated. So for instance if there is code somewhere that does from some_module import fast_function, or x = fast_function, those references to the function won't be affected by changing it in the module where it was originally defined. – Ballata

You are going about this the wrong way; what you should be doing is create unittests for that function and use those to debug just the function. – Burkhard

Answer 1

You can execute a single-line python statement inside pdb by preceding it with an exclamation sign. The output of help exec as produced by pdb follows:

(!) statement
        Execute the (one-line) statement in the context of the current
        stack frame.  The exclamation point can be omitted unless the
        first word of the statement resembles a debugger command.  To
        assign to a global variable you must always prefix the command
        with a 'global' command, e.g.:
        (Pdb) global list_options; list_options = ['-l']
        (Pdb)

Using this facility, you can save the source code of the function in a file and update that function inside pdb as follows:

!exec(open("fast_function.py", "r").read())

Demonstration:

$ cat test.py 
import pdb;

def foo():
    print('Foo');

foo()
pdb.set_trace()
foo()

$ python3 test.py 
Foo
> test.py(8)<module>()
-> foo()
(Pdb) cont
Foo

$ cat foo.py 
def foo():
    print('Modified Foo');

$ python3 test.py 
Foo
> test.py(8)<module>()
-> foo()
(Pdb) !exec(open("foo.py", "r").read())
(Pdb) cont
Modified Foo

Kunming

In order to use exec, module level statements should be protected with if __name__ == '__main__'. – Baskin

Answer 2

If it's a short function you can just overwrite the existing function in one pdb line. Note that the exclamation sign is optional there.

(pdb)...
(pdb)!def fast_function(something_else): print("Hello world");return True

If the function is a bit larger in code length then you can leverage the normal interactive shell(see this)

(Pdb) !import code; code.interact(local=vars())
(InteractiveConsole)
In : def fast_function(something_else):
...:     print 'hello in pdb'
...: 
In : # use ctrl+d here to return to pdb shell...
(Pdb) !fast_function(arg)
hello in pdb

If the code of the function is not easily manageable with the interactive shell either, Leons' recommendation will be better I guess.

Just keep in mind that, every pdb line can work like executing a line of normal python code and you can achieve pretty much anything on the fly! pdb is more powerful than some other graphical debugging tools in this sense.

P.S. seems like PyCharm does support Evaluating expression feature according to ADR

Neology

You can change function in Evaluating expression in PyCharm. jetbrains.com/help/pycharm/evaluating-expressions.html – Eugenides

I haven't tried yet, but would copy-pasting the fixed code into the interactive console work? – Urethrectomy

@EdgardDerby That would work as far as the fixed code doesn't include unnecessary (empty) newlines. The def block finishes as soon as the interactive shell meets empty line. – Neology

Answer 3

---

edit 19/01/2018

You can write result in a file on memory. for example /dev/shm is a tmpfs partition. size could be optimized with protocol dump's keyargs.

# save result
with open('/dev/shm/data.pk', 'w' ) as data:
    pickle.dump(something_else, data, protocole=3)

---

You can use pickle to store result in file the first time and reload it to debug second function

import pickle

def slow_function(something):
  # I'm a very slow function
  return something + 42

def fast_function(something_else):
  # There's a bug here
  return something_else + 42


something = 1

something_else = slow_function(something)

# save result
with open('data.pk', 'w' ) as data:
    pickle.dump(something_else, data)

second launch

import ipdb

# load result from disk
with open('data.pk', 'r' ) as data:
    something_else = pickle.load(data)

ipdb.set_trace()
fast_function(something_else)

Alternation

Thanks for providing this but as stated in the question I don't want to store the results on disk. I just want to update the changed function. – Urethrectomy