Microsoft Roslyn vs. CodeDom

Question

From a press release yesterday on InfoWorld regarding the new Microsoft Roslyn:

The most obvious advantage of this kind of "deconstructed" compiler is that it allows the entire compile-execute process to be invoked from within .Net applications. Hejlsberg demonstrated a C# program that passed a few code snippets to the C# compiler as strings; the compiler returned the resulting IL assembly code as an object, which was then passed to the Common Language Runtime (CLR) for execution. Voilà! With Roslyn, C# gains a dynamic language's ability to generate and invoke code at runtime.

I've been able to do this since the release of .NET 4 with CSharpCodeProvider.CompileAssemblyFromSource which I in fact use in an ASP.Net project written awhile ago that does exactly that - allows a user to type in code into a textbox, choose assemblies/namespaces to reference, and then execute and display the output from that code on-the-fly for live environment code testing on Windows Azure.

Is CodeDom part of / a precurser to Roslyn? What's the special benefit of Roslyn over CodeDom?

Portentous

Answer 1

Disclaimer: I work for Microsoft on the Roslyn team.

CodeDom is a precursor to Roslyn, but is only marginally related. Essentially, CodeDom is a simple and (somewhat) langage agnostic way to generate code that was added in .NET 1.0 to support designers (a la WinForms). Because CodeDom was an attempt at providing a unified model that can generate code in C#, VB, and other languages, it lacks high fidelity with any of the languages that it supports (that's why you can't create a switch statement with CodeDom). CSharpCodeProvider.CompileAssemblyFromSource is simply a wrapper around executing csc.exe.

Roslyn is a completely different animal. It is a rewrite of both the C# and VB compilers from the ground up using managed code -- C# in C# and VB in VB (the versions of csc.exe and vbc.exe that ship today are written in native code). The advantage of building them in managed code is that users can reference the real compilers as libraries from .NET applications (no wrappers needed).

While building each component of the compiler pipeline, we've exposed public APIs on top:

• Parser -> Syntax Tree API
• Symbol Table/Metadata Import -> Symbol API
• Binder -> Binding and Flow Analysis APIs
• IL Emitter -> Emit API

Roslyn can be used as a sophisticated C# and VB source code generator, but that's where the similarity to CodeDom ends. The Roslyn Compiler APIs can be used to parse code, perform semantic analysis, compile and evaluate code dynamically, etc.

In addition to the compilers, the Roslyn team is also rebuilding the Visual Studio C# and VB IDE features on top of the public compiler APIs. So, the compiler APIs are rich enough to build the Visual Studio design-time tools, like IntelliSense and the Extract Method refactoring. Also, at layers above the compiler, Roslyn offers services for higher-level analysis or data transformation. For example, there are services for formatting code using the C# and VB formatting rules, or finding all references to a particular symbol within a solution.

Really, there isn't just one special benefit of Roslyn over CodeDom. Where CodeDom filled a very specific code generation need, Roslyn is tackling the entire language tooling space by providing a framework to allow you to build just about any sort of C# or VB language tool you can think of.

Salicylate

@Dustin: Will Roslyn be supporting other languages? JavaScript(.NET), for example? – Declassify

@Dustin: This is perfect for building a complete IDE experience that can enforce code quality in my organization, although I don't see a complete replacement of manual code review but I see a considerable increase in quality. Soon! – Uniparous

It would be great if someone had already created a Roslyn-based tool to convert code that uses CodeDom into code that uses Roslyn's SyntaxFactory... (Partly because .Net Core has Roslyn but no CodeDom and I'm using a lib built around CodeDom) – Semilunar

Answer 2

CodeDom allows you to compile - but it doesn't give you the ability to really get information about the code itself (other than compiler errors). Basically, it's a black box where you say "compile this" and it says "I succeeded" or "I failed, here are some errors".

Roslyn allows you to completely inspect and build out the code on the fly. This includes things like being able to see/inspect the comments within a piece of source code, detailed information about the full structure, etc. You can go through and get the entire syntax tree of the source you pass into Roslyn, and do detailed analysis or transformations on it.

Given the full, rich syntax information, you have a huge amount of extra control and flexibility. This is how, for example, the sample works that copies a block of C# code and pastes it as VB.NET code. With Roslyn, you can do more than just compile - you can also manipulate the code itself cleanly. This should make a lot of tooling far simpler to generate, since things like refactorings can be done very simply as the tooling understands the full syntax, including meta information (like comments), and can just work with it directly.

Particle

Answer 3

One big difference I see: with CodeDom, each time you compile some C# or VB.NET, it happens out of process. CSC.exe or VBC.exe are the real workers behind the scene.

If you want to build a service, in terms of architecture, scalability, isolation, etc. (you mention Azure), this is not very good.

With Roslyn it's in process.

I suppose this is one of the reason they call it "Compiler as a service".

Also, CodeDom is a relatively poor API, misses a lot of features, and is not really up to date, as it was designed mostly to support Visual Studio UI designers automatic code generation. I think Roslyn will do much better as it's written by the guys who write the compilers. I hope that will make the difference.

PS: One notable difference from CSC.exe and VBC.exe: Roslyn seems to be pure .NET (and uses CCI).

Helldiver

Answer 4

Roslyn allows much much finer control of the whole process - for example you could analyse the string and even generate additional code (on-the-fly within the compile process based on the analysis), etc.

CodeDom is "just using the compiler" while Roslyn is "compiler as a service with full access to (sub-) parts"... with Roslyn you are "inside the compiler" and can see what the code looks like from a compiler perspective allowing you to change things in ways currently not possible.

For example, you can use Roslyn to extend C# - something very handy and much better than the current state of AOP implementation.

For an overview of the current Roslyn state and the different levels of access and control it provides, see http://msdn.microsoft.com/en-us/hh500769

UPDATE

Microsoft just made a new CTP available with additional features and lots of API changes/additions. For details see here.

Reimer

Actually, it isn't true that you can use Roslyn to extend C# with additional keywords. – Salicylate

thanks... corrected... although not in the first release I am pretty that this will be possible... – Reimer

@DustinCampbell, What if you handled whatever compiler error that the pseudo keyword caused by generating code? – Beliabelial

You'd need to do a rewrite before passing it to the compiler. First, parse code with your special keywords. The code will parse and, unless the parser couldn't make heads or tails of it, the invalid keywords will show up as SkippedTokenTrivia in the resulting tree. Then, detect the skipped keywords and rewrite the tree with valid code (e.g. AOP weaving). Finally, pass the new tree to the compiler. This is definitely a hack though, and it's not guaranteed to work with future versions of Roslyn. E.g. the parser might produce not produce the same tree for broken code in future releases. – Salicylate

@DustinCampbell: but will there be SOMETHING allowing AOP weaving in Roslyn final? My Mono.Cecil INPC weaving works just fine as is, but if I could write public notifying string Name {get;set;} that would be even more awesome – Cirrate

Answer 5

As of January 10th, 2022, use of Roslyn has a rather serious memory leak makes it all but unusable reported here https://github.com/dotnet/roslyn/issues/41722 and a separate, but possibly related issue of leaving file handles open reported here https://github.com/dotnet/roslyn/issues/42134. Both of these issues have been open for nearly two years.

So one very big difference between CodeDom versus Roslyn at the moment is that Roslyn suffers these issues making it unusable in a production environment where calls to compile code are done many times. CodeDom does not suffer these issues.

I just felt like this was important information to know for anyone who landed on this page that is considering a rewrite of existing code.

Plumbaginaceous