Is there a way to debug a subprocess using pydev?

Question

I'm using Eclipse / PyDev trying to find a way to debug code that uses subprocess.Popen to create a child process: I want to be able to debug the child process that is created. The problem is that I cannot find a way to debug accross process boundaries, and I'm guessing that it is actually not possible. Still, you never know until you ask, and so that I am doing!

A bit of background: I have a complex build process driven by Waf which invokes our unit tests by calling out to nose as required: I want to hook into these processes to debug unit test failures. I know I could try to run nose directly but the problem is that the environment I have to configure for our modules to load correctly is fairly complex and I don't want to duplicate the code to do that if I can avoid it.

I'm aware of the remote debugging mode but thats pretty inconvenient because I have to manually invoke the debugger in the remote process. If anyone knows a way to do what I'm trying to do it would be much appreciated.

Badlands

This question and its answers are quite old, but I'd like to suggest that you accept the answer from pimlottc instead of the currently accepted answer from Raphael. Raphael's answer may have been correct when it was given, but pimlottc's answer is absolutely correct and very useful now. – Charry

Answer 1

I does not seem PyDev can do it (neither can PyDbg and WinDbg), but it looks like gdb can: http://wiki.python.org/moin/DebuggingWithGdb.

Joannajoanne

Can it follow the fork into the subprocess or are you just suggesting attaching to the subprocess manually? Interesting to learn that GDB can inspect python stacks: I didn't know that :) – Badlands

The limitation of having to attach manually, by the way, is actually a result of how Unix systems work, and is not specific to any particular language or debugger. – Hardness

Yeah, thats true of course: its a shame there is no way to attach the a process and get the same kind of control as you do when it is your direct child with the Pydev debugger. – Badlands

Answer 2

I've found something of a workaround that might work for you.

Like you, I first found the remote debugging option of manually inserting calls to pydevd.settrace() at desired breakpoints. But I also noticed that subsequent PyDev breakpoints (i.e. those created by clicking in the left margin) were obeyed. So it seems that you just need the first explicit settrace call to establish the remote debugging session for the process, and afterwards just use the normal debugger breakpoints.

Moreover, you can modify the settrace call so it doesn't actually suspend the process:

import pydevd
pydevd.settrace(suspend=False)

So insert the above code somewhere early in the initialization of the subprocess and you should be good. Still a bit of a hack, but it's definitely better than the manual method.

Tenuto

There is also an additional flag for settrace to apply to subthreads, but it didn't seem to work that reliably for me: pydevd.settrace(suspend=False, trace_only_current_thread=False) – Tenuto

As of PyDev 1.6.4, tracing subthreads works fine for me. The pydev.settrace(...) call is placed in the main thread. – Simson