The DLR, Boo, and the JVM

Question

I'm just starting to try to learn more about the .Net VM underpinnings, and am immediately thrown off by something. I know there's this new thing called the DLR that allows for all the dynamic stuff in C# and the running of the IronX languages. But now I'm reading about this language called Boo, and apparently it has had dynamic capabilities since long before the DLR existed. So,

1) How is this even possible?

2) What does the DLR add to the equation?

3) Would a language like Boo stand to gain anything by re-implementing itself in terms of the DLR?

From what I've kind of gathered here and there, it looks like the DLR came out of IronPython, when they factored out everything that was necessary for DL support in .Net, and put it in reusable form. So what I'm guessing is that DLR is nothing special, just some libraries that help with dynamic objects in Microsoft.Scripting.dll, but nothing that you couldn't just go out and code up by yourself if you had the time, which I guess is what happened for Boo? And then for 2 and 3, I guess the commonality and reusability of the DLR would allow any future DLR improvements to be grandfathered in automatically, but that there's no urgent "need" to re-implement using the DLR if you've already made your own custom runtime? Or does the DLR have some secret MS sauce that makes it better than anything we could do on top of .Net?

4) Is the DLR really a runtime or just a set of libraries? (What exactly is a runtime anyway? I probably need to learn more compiler theory before I can even understand an answer to this question, or whether it's even a question that means anything. Ignore this question. Or don't.)

5) How does IronPython compilation work? Does it compile down to a new dynamic version of the CIL, or does it just prepend a "ironpython.exe" command to a string with the program's text in it? Hmm, well if dynamic is a keyword in C#, then there must be a dynamic version of the CIL, right? So how does .Net know whether to use the CLR or the DLR on the CIL?

6) Is the DaVinci project for the JVM different? It looks like it's an actual re-implementation of the JVM itself. What are the implications of this approach? I'm guessing there's huge performance gains, but anything else? Any reason MS didn't take this road?

7) Does the DLR make Boo somewhat obsolete for making DSLs?

Fishbein

In the future, please break down your questions in several posts. This will increase the probability of good answers from different people with specific knowledge about each one. – Broglie

Answer 1

The DLR basically brings 3 things to the party:

• An extended set of expression trees (first introduced w/ LINQ) that enable compiling complete programs. These provide a much easier way to generate code than generating IL directly - it gets rid of many cases of being able to generate invalid IL and turns many more cases into easily debuggable runtime exceptions.
• A built in call site caching mechanism so you don't need to create your own (very useful for good performance in dynamic languages). This includes things like a multi-level cache and aging out unused items.
• A meta object protocol which allows dynamic languages to talk to each other at runtime and negotiate the correct result for the calling language (for example returning undefined in JavaScript when a member doesn't exist or throwing a AttributeError in Python regardless of the language the dynamic object was written in).

The meta object protocol is the only piece that absolutely needs to be shared - everything else you could create on your own.

IronPython builds fully on top of the DLR - so it's compilation model is actually to compile to expression trees. The DLR inner layer which shipped w/ .NET 4.0 is used to compile those expression trees and we use the interpreter which is a piece of the outer layer to interpret those expression trees. We can then lazily compile the expression trees after the interpreted versions have become hot. That compilation includes the production of call sites which we use for dynamic dispatch of various operations (getting, setting members, calling objects, etc...) and again we use the DLR - in this case it's call site mechanism. IronPython uses a combination of both standard DLR binders for these operations along with custom binders which do IronPython specific actions (flow through code context, supporting *args and **args calls, etc...) which then fall back to standard DLR binders for interop.

The Davinci project will add "method handles" to the JVM which the CLR has already had in the form of delegates. It will also add a new "invokedynamic" opcode which the CLR doesn't have and didn't gain w/ the DLR work. Instead the DLR just uses the existing primitives (delegates, reified generics) plus libraries for defining the interop protocol. Both add the concept of call sites and those may be fairly similar between the two.

Waxbill

Answer 2

Lots of questions here! I'm not sure I can answer all of them, but I'll do as much as I can:

1. Boo is not dynamic in the same sense that (Iron)Python is. It's mostly a statically typed language with strong type inference and pythonic syntax. This, coupled with its optional duck typing, give it a very dynamic feel, but it's certainly not the same as Python. Boo is more similar (except for syntax) to C# 4 than Python.

2. DLR adds dynamic support for language implementors on top of the CLR, which is more geared towards statically typed languages (such as VB.NET, C# ,F#)

3. Not really IMHO. It would become too similar to IronPython. Precisely one of the characteristics of Boo is that it's statically typed.

4. Runtimes are libraries which support some basic constructs in the language. VB.NET, C#, F#, Boo, they all have runtime libraries. You usually never see VB.NET or C# runtimes because they come with the .NET framework. There was a great answer on SO about this from Eric Lippert but I can't find it.

5. Can't comment on this, don't have much hands-on experience with IronPython.

6. Don't know about the DaVinci project, can't comment on this.

7. No. As far as I know Boo's macros and extensible compiler are quite unique for a .NET language (Nemerle has similar macro capabilities). I can't really say if Boo DSLs can be more or less powerful than IronPython DSLs. What I can say for sure is that the implementation of Boo DSLs is wildly different from the implentation of Python DSLs.

Broglie