traceback shows only one line of a multiline command

Question

I have added a small debugging aid to my server. It logs a stack trace obtained from traceback.format_stack()

It contains few incomplete lines like this:

File "/home/...../base/loop.py", line 361, in run
    self.outputs.fd_list, (), sleep)

which is not that much helpfull.

The source lines 360 and 361:

rlist, wlist, unused = select.select(self.inputs.fd_list,
                                     self.outputs.fd_list, (), sleep)

If only one line can be part of the stack trace, I would say the line 360 with the function name (here select.select) is the right one, because the stack is created by calling functions.

Anyway, I would prefer the whole (logical) line to be printed. Or at least some context (e.g. 2 lines before). Is that possible? I mean with just an adequate effort, of course.

Tried to add a line continuation character \, but without success.

---

EPILOGUE: Based on Jean-François Fabre's answer and his code I'm going to use this function:

def print_trace():
    for fname, lnum, func, line in traceback.extract_stack()[:-1]:
        print('File "{}", line {}, in {}'.format(fname, lnum, func))
        try:
            with open(fname) as f:
                rl = f.readlines()
        except OSError:
            if line is not None:
                print("    " + line + "  <===")
            continue
        first = max(0, lnum-3)
        # read 2 lines before and 2 lines after
        for i, line in enumerate(rl[first:lnum+2]):
            line = line.rstrip()
            if i + first + 1 == lnum:
                print("    " + line + "  <===")
            elif line:
                print("    " + line)

Wolof

Can you post more lines of your stack trace? – Paly

@LaurentLAPORTE Other lines look very similar. Basically the whole stack trace is just those two line template repeating over and over. File "file", line N, in func in the first line and a corresponding source code line in the second line. – Wolof

Answer 1

"just with adequate effort" this can be done. But it's hack-like

check this example:

import traceback,re,os,sys

r = re.compile(r'File\s"(.*)",\sline\s(\d+)')

def print_trace():
    # discard the 2 deepest entries since they're a call to print_trace()
    lines = [str.split(x,"\n")[0] for x in traceback.format_stack()][:-2]

    for l in lines:
        m = r.search(l)
        if m != None:
            sys.stdout.write(l+"\n")
            file = m.group(1)
            line = int(m.group(2))-1
            if os.path.exists(file):
                with open(file,"r") as f:
                    rl = f.readlines()
                    tblines = rl[max(line-2,0):min(line+3,len(rl))]
                    # read 2 lines before and 2 lines after
                    for i,tl in enumerate(tblines):
                        tl = tl.rstrip()

                        if i==2:
                            sys.stdout.write("    "+tl+" <====\n")
                        elif tl:
                            sys.stdout.write("    "+tl+"\n")


def foo():
    print_trace()

foo()

output:

  File "C:\Users\dartypc\AppData\Roaming\PyScripter\remserver.py", line 63, in <module>
    if __name__ == "__main__":
        main() <====
  File "C:\Users\dartypc\AppData\Roaming\PyScripter\remserver.py", line 60, in main
        t = SimpleServer(ModSlaveService, port = port, auto_register = False)
        t.start() <====
    if __name__ == "__main__":
  File "C:\Program Files\PyScripter\Lib\rpyc.zip\rpyc\utils\server.py", line 227, in start
  File "C:\Program Files\PyScripter\Lib\rpyc.zip\rpyc\utils\server.py", line 139, in accept
  File "C:\Users\dartypc\AppData\Roaming\PyScripter\remserver.py", line 14, in _accept_method
    class SimpleServer(Server):
        def _accept_method(self, sock):
            self._serve_client(sock, None) <====
    class ModSlaveService(SlaveService):
  File "C:\Program Files\PyScripter\Lib\rpyc.zip\rpyc\utils\server.py", line 191, in _serve_client
  File "C:\Program Files\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py", line 391, in serve_all
  File "C:\Program Files\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py", line 382, in serve
  File "C:\Program Files\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py", line 350, in _dispatch
  File "C:\Program Files\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py", line 298, in _dispatch_request
  File "C:\Program Files\PyScripter\Lib\rpyc.zip\rpyc\core\protocol.py", line 528, in _handle_call
  File "<string>", line 420, in run_nodebug
  File "C:\DATA\jff\data\python\stackoverflow\traceback_test.py", line 31, in <module>
        print_trace()
    foo() <====

EDIT: VPfB suggested the use of extract_stack which is a little less "hacky", no need to parse a string, just get the quadruplet with traceback info (needs to rebuild the text message, but that's better)

import traceback,os,sys


def print_trace():
    # discard the 2 deepest entries since they're a call to print_trace()

    for file,line,w1,w2 in traceback.extract_stack()[:-2]:
        sys.stdout.write('  File "{}", line {}, in {}\n'.format(file,line,w1))

        if os.path.exists(file):
            line -= 1
            with open(file,"r") as f:
                rl = f.readlines()
                tblines = rl[max(line-2,0):min(line+3,len(rl))]
                # read 2 lines before and 2 lines after
                for i,tl in enumerate(tblines):
                    tl = tl.rstrip()

                    if i==2:
                        sys.stdout.write("    "+tl+" <====\n")
                    elif tl:
                        sys.stdout.write("    "+tl+"\n")


def foo():
    print_trace()

foo()

Kriegspiel

Just on the edge with the adequateness :-) I'd use extract_stack to get the file and line without regexps, but +1 for the idea. – Wolof

added alternate code, thanks for the tip (and the +1 :)). – Beveridge

I have appended similar code to the question. if i==2: is not correct if line number is 1 or 2. – Wolof

you're right. Never trust indices, I should have known :) A dictionary with real line numbers would be better, also, a global file cache could be more efficient to avoid reading the file fully multiple times if the callstack is mainly in a single file. But now you got hold of the code, you can improve it and I don't even claim a GPL on it :) – Beveridge

Great one. Its very useful. – Decay

Answer 2

The traceback.format_exception_only function format only one line, except in case of SyntaxError, so…

Paly