Writing an Iron Python debugger

Question

As a learning exercise I'm writing myself a simple extension / plugin / macro framework using IronPython - I've gotten the basics working but I'd like to add some basic debugging support to make my script editor easier to work with.

I've been hunting around on the internet a bit and I've found a couple of good resources on writing managed debuggers (including Mike Stall's excellent .Net Debugging blog and the MSDN documentaiton on the CLR Debugging API) - I understand that IronPython is essentially IL however apart from that I'm a tad lost on how to get started, in particular:

• Are there any significant differences between debugging a dynamic language (such as IronPython) to a static one (such as C#)?
• Do I need to execute my script in a special way to get IronPython to output suitable debugging information?
• Is debugging a script running inside the current process going to cause deadlocks, or does IronPython execute my script in a child process?
• Am I better off looking into how to produce a simple C# debugger first to get the general idea?

(I'm not interested in the GUI aspect of making a debugger for now - I've already got a pretty good idea of how this might work)

Ayo

This question answers one of my questions: #676688 – Ayo

Answer 1

A few more links have started to make this a lot clearer - there are 2 ways of adding debugger support that I've seen:

Debugging IronPython as a CLR application

The first is to use the fact that IronPython emits IL and debug it using standard techniques used for debugging .Net apps. There is a series of blog posts on this approach by Harry Pierson here about the development of ipydbg - a python debugger that uses this approach.

• See this post for an overview on where .Net debugging functionality is exposed, and the various wrappers around it (mdbg)
• The disadvantage of this approach is that this form of debugging completely blocks the application being debugged, and so you must execute your scripts in a second application.

Using Microsoft.Scripting.Debugging

Because of this limitation the Microsoft.Scripting.Debugging library was produced instead which is far more suited to for applications which run IronPython "embedded" (i.e. in the same process).

There is an introduction to it here and a more extensive blog post on how it used here - essentially it consists of a callback function which is executed every time anything "interesting" happens (every time we enter a function, every time we return from a function and every time a line is executed). Execution of the script is blocked while the callback function is running, which allows you to "break" the script.

I've decided to take the second approach - i'll update this post as I find more information that might be helpful to other trying to do this.

Ayo

Answer 2

I am no expert in IronPython, but I have some experience in debugging managed applications using WinDbg. I looked briefly at how IronPython applications look in the debugger. Due to the dynamic nature of Python a lot of code is emitted at runtime. This makes debugging an IronPython application somewhat more complicated than say a C# application because you have an additional layer of code generation so to speak.

Harry Pierson who used to be heavily involved in the development of the Iron-languages has a series of blog post on writing an IronPython debugger, that has a lot of details.

Bartonbartosch

Thanks for the link - I'm reading it through now and it looks good. – Ayo

Answer 3

At AlterNET Software (www.alternetsoft.com) we have implemented debugging both for C# and IronPython, and we have found that the major difference that for C# debugging you can't really have debugger and script being debugged to be part of the same process. We tried few techniques described here, but could not really make it to work in reliable way.

https://learn.microsoft.com/en-us/archive/blogs/jmstall/you-cant-debug-yourself

So in order for your scripts to be debugged, you would either need to use a standalone debugger, or to write scripts in such a way they can be compiled in the separate executable (but this would mean that in order to access application-define objects such script will need to interact with the application using interprocess-communication techniques). You can find a bit more information here: https://www.alternetsoft.com/news/known-issues-of-net-script-executing-and-debugging

Dynamic languages, such as IronPython are a lot more permissive in this sense - you can implement debugging in your own process.

The script being debugged would need to be run in the separate thread though, so if it needs to access application objects, this should be done in thread-safe way - like by using synchronized calls to the UI thread with Invoke. Potentially it can be avoided with implementing own message loop during debugging, but we haven't tried it (yet).

Dmitry

Tenner