Python Decorator for printing every line executed by a function

Question

I want to, for debugging purposes, print out something pertaining to each and every line executed in a python method.

For example if there was some assignment in the line, i want to print what value was assigned for that variable, and if there was a function call, i want to print out the value returned by the function, etc.

So, for example if i were to use a decorator, applied on function/method such as :

@some_decorator
def testing() : 
    a = 10
    b = 20
    c = a + b
    e = test_function()

the function testing when called, should print the following :

a = 10
b = 20  
c = 30
e = some_value

Is there some way to achieve this? More fundamentally, i want to know whether i can write a code that can go through some other code line by line, check what type of an instruction it is, etc. Or maybe like we can get a dictionary for finding out all the variables in a class, can i get a dictionary like datastructure for getting every instruction in a function, which is as good a metaprogram can get.

Hence, I am particularly looking a solution using decorators, as I am curious if one can have a decorator that can go through an entire function line by line, and decorate it line by line, but any and all solutions are welcome.

Thanks in advance.

Merrymaker

It could be done, but I don't know of one. Essentially, you can extract the file and line number from the decorated function, go re-read the function, compile it to an AST, insert nodes in the AST, and then compile the AST and use that as the function. An interesting project. – Corelli

Maybe the pdb module will be helpful? – Briggs

@perreal could you please explain your approach in a bit more detail or provide a link where I could find more about the elements in your approach, and if possible how to apply these to my problem. thanks by the way for your reply – Merrymaker

@ironstein: check this out for starters tomforb.es/automatically-inline-python-function-calls – Driftage

Basically you have to wrap your function in a function that loads it into a debugger and then single step through the function. – Skittish

@Skittish could you please be more specific on how do I do this, as I am not familiar with using a debugger – Merrymaker

What's "a line"? You can have an expression occupying multiple lines of text and have more than one statement in a line a=1;b=2. Note that python compiles to bytecode and a "line" can produce any number of bytecodes... the sys module allow tracing through function calls, but I believe there is no easy way to get execution exactly by lines. – Cilo

@Cilo you are right. What I was trying to say was one expression – Merrymaker

Answer 1

How about something like this? Would this work for you?

Debug Context:

import sys

class debug_context():
    """ Debug context to trace any function calls inside the context """

    def __init__(self, name):
        self.name = name

    def __enter__(self):
        print('Entering Debug Decorated func')
        # Set the trace function to the trace_calls function
        # So all events are now traced
        sys.settrace(self.trace_calls)

    def __exit__(self, *args, **kwargs):
        # Stop tracing all events
        sys.settrace = None

    def trace_calls(self, frame, event, arg): 
        # We want to only trace our call to the decorated function
        if event != 'call':
            return
        elif frame.f_code.co_name != self.name:
            return
        # return the trace function to use when you go into that 
        # function call
        return self.trace_lines

    def trace_lines(self, frame, event, arg):
        # If you want to print local variables each line
        # keep the check for the event 'line'
        # If you want to print local variables only on return
        # check only for the 'return' event
        if event not in ['line', 'return']:
            return
        co = frame.f_code
        func_name = co.co_name
        line_no = frame.f_lineno
        filename = co.co_filename
        local_vars = frame.f_locals
        print ('  {0} {1} {2} locals: {3}'.format(func_name, 
                                                  event,
                                                  line_no, 
                                                  local_vars))

Debug Decorator:

def debug_decorator(func):
    """ Debug decorator to call the function within the debug context """
    def decorated_func(*args, **kwargs):
        with debug_context(func.__name__):
            return_value = func(*args, **kwargs)
        return return_value
    return decorated_func

Usage

@debug_decorator
def testing() : 
    a = 10
    b = 20
    c = a + b

testing()

Output

###########################################################
#output:
#   Entering Debug Decorated func
#     testing line 44 locals: {}
#     testing line 45 locals: {'a': 10}
#     testing line 46 locals: {'a': 10, 'b': 20}
#     testing return 46 locals: {'a': 10, 'b': 20, 'c': 30}
###########################################################

Signory

This is soooo handy, this should be marked as correct answer. – Bashemeth

How do we extend this for class methods ? – Bowls

How do you print the code from the line ? – Formulaic

This is really cool feature, have no idea, why there's no library with such functionality... Btw is it possible to filter tracing lines by some conditions, e.g. if there's a loop, then trace only start of it, because otherwise it will log each loop iteration – Algophobia

There is an issue in 3.6 with the line sys.settrace = None it needs to read sys.settrace(None) -- other than that, this is amazing! Thank you! – Allseed

This is quite possible the most useful piece of code I have found, It will save me hours in the debugging process, easily customizable to meet specific needs. – Lor

Answer 2

If you add:

...
co = frame.f_code
if self.filename is None:
    self.filename = co.co_filename
cur_filename = co.co_filename
if cur_filename == self.filename:
    ...

to def trace_line you can keep to only tracing the current file

Lor