Dynamic code generation

Question

I am currently developing an application where you can create "programs" with it without writing source code, just click&play if you like.

Now the question is how do I generate an executable program from my data model. There are many possibilities but I am not sure which one is the best for me. I need to generate assemblies with classes and namespace and everything which can be part of the application.

1. CodeDOM class: I heard of lots of limitations and bugs of this class. I need to create attributes on method parameters and return values. Is this supported?

2. Create C# source code programmatically and then call CompileAssemblyFromFile on it: This would work since I can generate any code I want and C# supports most CLR features. But wouldn't this be slow?

3. Use the reflection ILGenerator class: I think with this I can generate every possible .NET code. But I think this is much more complicated and error prone than the other approaches?

4. Are there other possible solutions?

EDIT: The tool is general for developing applications, it is not restricted to a specific domain. I don't know if it can be considered a visual programming language. The user can create classes, methods, method calls, all kinds of expressions. It won't be very limitating because you should be able to do most things which are allowed in real programming languages. At the moment lots of things must still be written by the user as text, but the goal at the end is, that nearly everything can be clicked together.

Alee

Can you give example of the type(s) of "programs" that will be created from this? – Deflective

I still can't really see what you are trying to achieve- is it somethign like a DSL or are you trying to create a general-purpose very-high-level programming language running on C#? – Fustigate

Yes - a general purpose high-level-programming language running on .NET (not necessarily c#). It is not very higher level than c# since I do not abstract any complexity/concepts away, instead I abstract syntax away. – Alee

Answer 1

You my find it is rewarding to look at the Dynamic Language Runtime which is more or less designed for creating high-level languages based on .NET.

It's perhaps also worth looking at some of the previous Stack Overflow threads on Domain Specific Languages which contain some useful links to tools for working with DSLs, which sounds a little like what you are planning although I'm still not absolutely clear from the question what exactly your aim is.

Fustigate

In particular, I would recommend to look at the syml.doc (dlr.codeplex.com/Project/Download/…) on the DLR Web site. It is actually a walktrough for creating your own simple language using Expression Trees and DLR. – Collazo

Answer 2

Most things "click and play" should be simple enough just to stick some pre-defined building-block objects together (probably using interfaces on the boundaries). Meaning: you might not need to do dynamic code generation - just "fake it". For example, using property-bag objects (like DataTable etc, although that isn't my first choice) for values, etc.

Another option for dynamic evaluation is the Expression class; especially in .NET 4.0, this is hugely versatile, and allows compilation to a delegate.

Flesh

Although its just click and play but the users can create real classes and members for it, I also need good performance, so I have to create "real" code. With expressions I can only generate Expressions, so I have to use it together with codeDOM? – Alee

or with dynamic assemblies. I've done lots of dynamic code, but I've never used codeDOM – Flesh

So you suggest Expression and Dynamic assemlies as replacement for codeDOM? Will this result in similar (runtime-)performance? Do you have a weblink on how to generate assemblies this way? – Alee

Expression is more for individual methods, but: InfoQ: infoq.com/articles/expression-compiler, or my blog: marcgravell.blogspot.com/search/label/expression – Flesh

Very interesting, but you say in your blog that Expressions with statement bodies won't be possible before .NET 4.0. Just Expressions won't be enough for me since I need control structures, try/catch/finally statements and the whole bunch. – Alee

Answer 3

Do the C# source generation and don't care about speed until it matters. The C# compiler is quite quick.

Persevering

My autocompletion feature will also rely on the compiled assembly, so it has be be kind of realtime ^^ – Alee

You could compile each class into its own assembly. – Persevering

Answer 4

When I wrote a dynamic code generator, I relied heavily on System.Reflection.Emit.

Basically, you programatically create dynamic assemblies and add new types to them. These types are constructed using the Emit constructs (properties, events, fields, etc..). When it comes to implementing methods, you'll have to use an ILGenerator to pump out MSIL op-codes into your method. That sounds super scary, but you can use a couple of tools to help:

• A pre-built sample implementation
• ILDasm to inspect the op-codes of the sample implementation.

Unsling

Answer 5

It depends on your requirements, CodeDOM would certainly be the best fit for a "program" stored it in a "data model".

However its unlikely that using option 2 will be in any way measurably slower in comparision with any other approach.

Remember

But if source generation and compilation isn't slower than codeDOM, Iam asking myself why something would need codeDOM at all, given that it is very buggy and limited. – Alee

I'd be interested in your source of information on its being "very buggy" and limited. One of the purposes of CodeDOM is to allow designers to create code using a single model and then the CodeDOM is converted to VB or CS or whatever language the host project is using. CodeDOM is also used in things like the Workflow Foundation to create a language independant Rules file. – Remember

See here: social.msdn.microsoft.com/Forums/en-US/netfxbcl/thread/… – Alee

Answer 6

I would echo others in that 1) the compiler is quick, and 2) "Click and Play" things should be simple enough so that no single widget added to a pile of widgets can make it an illegal pile.

Good luck. I'm skeptical that you can achieve point (2) for anything but really toy-level programs.

Alkmaar