ILGenerator method inlining

Question

Given following code:

using System;
using System.Reflection.Emit;
using System.Diagnostics;
using System.Reflection;

namespace ConsoleApplication1
{
    class A
    {
        public int Do(int n)
        {
            return n;
        }
    }

    public delegate int DoDelegate();

    class Program
    {
        public static void Main(string[] args)
        {
            A a = new A();

            Stopwatch stopwatch = Stopwatch.StartNew();
            int s = 0;
            for (int i = 0; i < 100000000; i++)
            {
                s += a.Do(i);
            }

            Console.WriteLine(stopwatch.ElapsedMilliseconds);
            Console.WriteLine(s);


            DynamicMethod dm = new DynamicMethod("Echo", typeof(int), new Type[] { typeof(int) }, true);
            ILGenerator il = dm.GetILGenerator();

            il.Emit(OpCodes.Ldarg_0);
            il.Emit(OpCodes.Ret);

            DynamicMethod dm2 = new DynamicMethod("Test", typeof(int), new Type[0]);
            il = dm2.GetILGenerator();


            Label loopStart = il.DefineLabel();
            Label loopCond = il.DefineLabel();

            il.DeclareLocal(typeof(int));   // i
            il.DeclareLocal(typeof(int));   // s

            // s = 0;
            il.Emit(OpCodes.Ldc_I4_0);
            il.Emit(OpCodes.Stloc_1);

            // i = 0;
            il.Emit(OpCodes.Ldc_I4_0);
            il.Emit(OpCodes.Stloc_0);

            il.Emit(OpCodes.Br_S, loopCond);

            il.MarkLabel(loopStart);

            // s += Echo(i);
            il.Emit(OpCodes.Ldloc_1);   // Load s
            il.Emit(OpCodes.Ldloc_0);   // Load i
            il.Emit(OpCodes.Call, dm);  // Call echo method
            il.Emit(OpCodes.Add);
            il.Emit(OpCodes.Stloc_1);

            // i++
            il.Emit(OpCodes.Ldloc_0);
            il.Emit(OpCodes.Ldc_I4_1);
            il.Emit(OpCodes.Add);
            il.Emit(OpCodes.Stloc_0);

            il.MarkLabel(loopCond);

            // Check for loop condition
            il.Emit(OpCodes.Ldloc_0);
            il.Emit(OpCodes.Ldc_I4, 100000000);
            il.Emit(OpCodes.Blt_S, loopStart);

            il.Emit(OpCodes.Ldloc_1);
            il.Emit(OpCodes.Ret);


            DoDelegate doDel = (DoDelegate)dm2.CreateDelegate(typeof(DoDelegate));
            s = doDel.Invoke();     // Dummy run to force JIT


            stopwatch = Stopwatch.StartNew();
            s = doDel.Invoke();
            Console.WriteLine(stopwatch.ElapsedMilliseconds);
            Console.WriteLine(s);
        }
    }
}

Call to method Do gets inlined. Loop finishes in about 40 ms. If I, for example, make Do to be virtual function, it doesn't get inlined, and the loop finishes in 240 ms. So far so good. When I use ILGenerator to generate Do method (Echo), and then generate DynamicMethod with the same loop as the given main method, call to Echo method never gets inlined, and it takes about 240 ms for a loop to finish. MSIL code is correct since it returns the same result as the C# code. I was sure that method inlining is something that is done by the JIT, so I see no reason for it not to inline the Echo method.

Does anybody know why this simple method wont get inlined by the JIT.

Jolty

Are you also generating the code which calls the dynamically generated Do() method, or is that code known at compile time? – Condescension

Could you include the full code sample which uses ILGenerator? And, just to be sure: are you testing under release build without the debugger attached? – Stratocumulus

I have reedited the post, giving the full code for the test app. I use release build and run it without the debugger. C# for loop inlines the method call and runs significantly faster than the IL for loop. – Jolty

Answer 1

After further investigation I have concluded following:

1. ILGenerator generated methods will never get inlined. It doesn't matter whether you call them using delegate, from another DynamicMethod or from a method created with MethodBuilder.
2. Existing methods (the ones coded in C# and compiled by VS) can get inlined only when called from a method created with MethodBuilder. If called from DynamicMethod, they will never get inlined.

I have concluded this after thoroughly testing many samples, and looking into final assembler code.

Jolty

Until someone shows differently, I think this is an accurate conclusion. Not inlining Dynamic methods would make sense sometimes (which is what I suggested in my response). Perhaps the compiler designers decided that it would be simplest to treat this as a rule for all cases. I wonder if expression trees behave the same way: msdn.microsoft.com/en-us/library/bb397951.aspx – Condescension

That would be interesting to check, though I didn't have time (or need) to play with expression trees. – Jolty

Sure this also includes IL-code generated and saved as dll and loaded after saving? Because i'm sure it not includes generated assemblies which has been saved. Anyway if your answer is correct you could just mark it as correct. – Communicative

Answer 2

If I'm understanding correctly, my guess is that since the method is generated dynamically, the JIT compiler doesn't know to inline it for callers.

I have written a great deal of IL but I haven't looked into inlining behavior (mainly because dynamic methods are usually fast enough for my purposes without further optimization).

I would welcome someone more knowledgeable on the subject to give feedback (please don't just downvote; I would like to learn something here if I am wrong).

Non-Dynamic

• you write "normal" .NET code (e.g. C#, VB.NET, any CLS-aware language)
• IL is created at compile time
• machine code is created at runtime; methods are inlined where appropriate

Dynamic

• you write "normal" .NET code whose purpose is to create a dynamic method
• IL is created at compile time for this code, but the dynamic method is not created
• machine code is created at runtime for the code which generates the dynamic method
• when that code is invoked, the dynamic method is created as IL in a special assembly
• dynamic method's IL is compiled to machine code
• however, JIT compiler doesn't recompile other callers to inline the new dynamic method. Or perhaps the other callers are dynamic themselves and haven't yet been created.

Condescension

I have added the fulle sample code that reproduces the problem. As you can see, i generate two dynamic methods. One is extrimly simple, and just return the same value it got as a param. The other method runs a simple for loop (the same one as in C# code at the begining). I do not expect JIT to inline the call to second method because it is called via a delegate. Howerer, I do expect him to inline the first method, when it is called inside the second method. When I inline it myself, I get 10 times faster execution of the same loop. – Jolty

Answer 3

You might try generating a dynamic assembly. My understanding is that most of the runtime is not aware that it is dynamic. I think internally it gets loaded like any other byte[] assembly. Therefore the JIT/inliner might also not be aware of it (or won't care).

Travelled

I have even tried that. Somehow and for some reason, dynamically generated method (no matter how you generate it) wont get inlined. Thanks anyway. – Jolty