Compile and run dynamic code, without generating EXE?

Question

I was wondering if it was possible to compile, and run stored code, without generating an exe or any type of other files, basically run the file from memory.

Basically, the Main application, will have some stored code (code that will potentially be changed), and it will need to compile the code, and execute it. without creating any files.

creating the files, running the program, and then deleting the files is not an option. the compiled code will need to be ran from memory.

code examples, or pointers, or pretty much anything is welcome :)

Acariasis

Answer 1

using (Microsoft.CSharp.CSharpCodeProvider foo = 
           new Microsoft.CSharp.CSharpCodeProvider())
{
    var res = foo.CompileAssemblyFromSource(
        new System.CodeDom.Compiler.CompilerParameters() 
        {  
            GenerateInMemory = true 
        }, 
        "public class FooClass { public string Execute() { return \"output!\";}}"
    );

    var type = res.CompiledAssembly.GetType("FooClass");

    var obj = Activator.CreateInstance(type);

    var output = type.GetMethod("Execute").Invoke(obj, new object[] { });
}

This compiles a simple class from the source code string included, then instantiates the class and reflectively invokes a function on it.

Pansy

GenerateInMemory = true doesn't do what its name suggests; it generates an assembly in the temp directory that you have to delete once you're done. – Partisan

@Tim: I'll admit to not being intimately acquainted with the CodeDom, but I haven't been able to find any evidence to back this up (on my hard drive in the temp directory or the application directory, nor on the web in a quick search). Could you point me to a resource that can confirm this? – Pansy

OK - last time I did this I must have kept the assemblies on disk for debugging. I can see that CSharpCodeProvider compiles to a temporary file, loads that file using Assembly.Load(byte[]), then deletes it. (It can't bypass the temp file completely because it runs csc.exe underneath.) – Partisan

@Adam, Tim: I ideally would want a system that would not create an assembly at all. Is there anyway to get CSharpCodeProvider to not create the assembly? – Acariasis

No, it's a fundamental limitation in the csc.exe compiler. Ironically, the compiler is written in native C++, and can't interact with your program. – Partisan

@AdamRobinson Are there any packages needed to be imported? Any special security settings or permissions needed? – Centesimo

Answer 2

Here is an example of how to use System.Linq.Expressions to add to Tim's answer. Obviously it isn't the prettiest code but having it in this nice tree-like form makes development so easy.

private  Expression<IsWordChar> CreateIsWordCharExpression()
{
    var e = Expression.Parameter(typeof(int), "e");
    var c = Expression.Variable(typeof(char), "c");
    var returnLabel = Expression.Label(Expression.Label(typeof(bool)), _falseConstant);
    var lambda = Expression.Lambda<IsWordChar>(
        Expression.Block(
            new[] { c },
            Expression.IfThen(
                Expression.OrElse(
                    Expression.Equal(e, Expression.Constant(-1)),
                    Expression.Equal(e, _inputLengthVar)
                ),
                Expression.Return(returnLabel.Target, _falseConstant)
            ),
            Expression.Assign(c, Expression.MakeIndex(_str, _stringCharsPropertyInfo, new[] { e })),
            Expression.IfThenElse(
                Expression.OrElse(
                    Expression.OrElse(
                        Expression.OrElse(
                            Expression.AndAlso(
                                Expression.GreaterThanOrEqual(c, Expression.Constant('a')),
                                Expression.LessThanOrEqual(c, Expression.Constant('z'))
                            ),
                            Expression.AndAlso(
                                Expression.GreaterThanOrEqual(c, Expression.Constant('A')),
                                Expression.LessThanOrEqual(c, Expression.Constant('Z'))
                            )
                        ),
                        Expression.AndAlso(
                            Expression.GreaterThanOrEqual(c, Expression.Constant('0')),
                            Expression.LessThanOrEqual(c, Expression.Constant('1'))
                        )
                    ),
                    Expression.Equal(c, Expression.Constant('_'))
                ),
                Expression.Return(returnLabel.Target, _trueConstant),
                Expression.Return(returnLabel.Target, _falseConstant)
            ),
            returnLabel
        ),
        "IsWordChar",
        new[] { e }
    );
    return lambda;
}

Impulsion

Answer 3

It's possible. It's easy or hard, depending on how much and what kind of code you want to write.

• Compile one-line expressions: System.Linq.Expressions (straightforward but limited)
• Compile whole programs: System.Reflection.Emit (hard to debug, but not impossible)

Edit: Note that, prior to .NET 4.0, System.Linq.Expressions is limited to what you can fit on a single line of C#: that is, no if, while, variable assignment etc.

Partisan

Don't be so quick to say System.Linq.Expressions is limited. Although it doesn't have the power of System.Reflection.Emit I have compiled massive programs with it. The ability to view the expression tree in a debug view is extremely helpful. – Impulsion

I like System.Linq.Expressions a lot :). It's just that, prior to .NET 4.0, it's limited to what you can fit on a single line of C# (i.e. no if, while, variable assignment etc.) – Partisan

@Tim - Oh that makes more sense. I have only used it in .NET 4.0. – Impulsion

There is a third alternative - Microsoft.CSharp. – Burg

@Tim: would you mind providing a simple code example? because ive been playing with options but they mostly seem to generate an assembly and I'm not sure how this is any different... (im mostly interested in compiling whole programs, so, system.reflection.emit?) – Acariasis

F# not C#, but it demonstrates the principle: partario.com/blog/2009/06/lisp-compiler-in-f-il-generation.html – Partisan

@Tim - Rather nicely I might add. It is awesome to see that F# is being used more and more. – Impulsion

Answer 4

Yes, you can do this. It is very slow, but you can certainly do it. Look at the CodeDOM or the (new CSharpCodeProvider().CreateCompiler()) in .Net.

Fields

Answer 5

Look into System.CodeDom. It will do exactly what you are looking for.

Amphimacer

System.CodeDom is the most pleasant option, but it still invokes the compiler and generates an assembly in the temp folder – Partisan

Answer 6

In Mono you use CSharp.Evaluator. It works truly in memory v. some of the other solutions mentioned that write out and read back in a file under the hood.

Oakland

Answer 7

Also take a look at embedding a scripting language like Python, Ruby, Lua, etc.. all of those support executing code from memory without anything being written to the disk.

Tegan

This should've been a comment. – Acariasis

I'd consider it an answer. (See mine below.) – Treponema

Answer 8

It's difficult, if not impossible to compile and execute C# without creating a file, because... Well, that's what compilation is- turning a language into an executable file. What you're looking for is some kind of scripting functionality. What you described in your question is essentially the difference between an interpreted language and a compiled language. See Wikipedia: Scripting language.

Depending on what you'll be using this feature for, you could make great use of a scripting language like Python, Ruby, or Lua. Here's an example: How do I run a Python script from C#?

This would make your application dependant upon python.exe (or whatever executable you'd need to run the scripting language you choose). If you want to avoid that, it might not be too hard to make your own scripting language that your application does the runtime for, depending on what you need to do with your injected script.

Edit: That first paragraph is rubbish. Sorry. See http://msdn.microsoft.com/en-us/library/8ffc3x75%28v=vs.110%29.aspx

Treponema

Thank you for this. C# does get compiled to MSIL, and I assume the way that .Net would handle MSIL is very similar to how other interpreted languages handle this. Creating a dependency to another runtime was impossible for me, and creating my own scripting language was too much work for relevance of the application. I ended up just writing the IL code and emitting it. That way I essentially get a scripting language that can be ran by the .Net runtime & I can convert back and forth to C# if needed (although it almost never is). – Acariasis

Thats nonsense. Compilation does not automatically mean to disk. many systems compile code in memory and then store it there for execution. – Oakland

@user430788, you're right, but I was referring to doing this with C#, which is not possible (see the comments on the question marked as answer- even compiling "in memory" uses csc.exe and creates a temp file). Well, I suppose it's possible, but you'd need something to process C# the way csc.exe does, and then execute it without making a file. Essentially a homebrew csc.exe, which would be incredibly more complicated than just using some kind of scripting. It would be quite impressive, though :) – Treponema

Mono CSharp.Evaluator compiles completely in memory. – Oakland