Performance of delegate and method group

Question

I was investigating the performance hit of creating Cachedependency objects, so I wrote a very simple test program as follows:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Web.Caching;

namespace Test
{
    internal class Program
    {
        private static readonly string[] keys = new[] {"Abc"};
        private static readonly int MaxIteration = 10000000;

        private static void Main(string[] args)
        {
            Debug.Print("first set");
            test2();
            test3();
            test4();
            test5();
            test6();
            test7();
            Debug.Print("second set");
            test7();
            test6();
            test5();
            test4();
            test3();
            test2();
        }

        private static void test2()
        {
            DateTime start = DateTime.Now;
            var list = new List<CacheDependency>();
            for (int i = 0; i < MaxIteration; i++)
            {
                list.Add(new CacheDependency(null, keys));
            }

            Debug.Print("test2 Time: " + (DateTime.Now - start));
        }

        private static void test3()
        {
            DateTime start = DateTime.Now;
            var list = new List<Func<CacheDependency>>();
            for (int i = 0; i < MaxIteration; i++)
            {
                list.Add(() => new CacheDependency(null, keys));
            }

            Debug.Print("test3 Time: " + (DateTime.Now - start));
        }

        private static void test4()
        {
            var p = new Program();
            DateTime start = DateTime.Now;
            var list = new List<Func<CacheDependency>>();
            for (int i = 0; i < MaxIteration; i++)
            {
                list.Add(p.GetDep);
            }

            Debug.Print("test4 Time: " + (DateTime.Now - start));
        }

        private static void test5()
        {
            var p = new Program();
            DateTime start = DateTime.Now;
            var list = new List<Func<CacheDependency>>();
            for (int i = 0; i < MaxIteration; i++)
            {
                list.Add(() => { return p.GetDep(); });
            }

            Debug.Print("test5 Time: " + (DateTime.Now - start));
        }

        private static void test6()
        {
            DateTime start = DateTime.Now;
            var list = new List<Func<CacheDependency>>();
            for (int i = 0; i < MaxIteration; i++)
            {
                list.Add(GetDepStatic);
            }

            Debug.Print("test6 Time: " + (DateTime.Now - start));
        }

        private static void test7()
        {
            DateTime start = DateTime.Now;
            var list = new List<Func<CacheDependency>>();
            for (int i = 0; i < MaxIteration; i++)
            {
                list.Add(() => { return GetDepStatic(); });
            }

            Debug.Print("test7 Time: " + (DateTime.Now - start));
        }

        private CacheDependency GetDep()
        {
            return new CacheDependency(null, keys);
        }

        private static CacheDependency GetDepStatic()
        {
            return new CacheDependency(null, keys);
        }
    }
}

But I can't understand why these result looks like this:

first set
test2 Time: 00:00:08.5394884
test3 Time: 00:00:00.1820105
test4 Time: 00:00:03.1401796
test5 Time: 00:00:00.1910109
test6 Time: 00:00:02.2041261
test7 Time: 00:00:00.4840277
second set
test7 Time: 00:00:00.1850106
test6 Time: 00:00:03.2941884
test5 Time: 00:00:00.1750100
test4 Time: 00:00:02.3561347
test3 Time: 00:00:00.1830105
test2 Time: 00:00:07.7324423

In particular:

1. Why is test4 and test6 much slower than their delegate version? I also noticed that Resharper specifically has a comment on the delegate version suggesting change test5 and test7 to "Convert to method group". Which is the same as test4 and test6 but they're actually slower?
2. I don't seem a consistent performance difference when calling test4 and test6, shouldn't static calls to be always faster?

Cimon

Answer 1

In tests with method group (4,6) C# compiler doesn't cache delegate (Func) object. It creates new every time. In 7 and 5 it caches Action object to generated method that calls your methods. So creation of Funcs from method groups is very slow (coz of Heap allocation), but calling is fast as action points directly on you method. And creation of actions from lambdas is fast as Func is cached but it points to generated method so there is one unnecessary method call.

Beware that not all lambdas can be cached (closures break this logic)

Cutup

Answer 2

In C# 11 the language spec was changed to allow the compiler to legally cache the delegate.

https://github.com/dotnet/roslyn/issues/5835

If you're using that version of C# or newer, you won't see allocations when passing a method group where the delegate can be cached.

Treadmill

Answer 3

I haven't looked too far into your code, but first step would be to switch things over to use the StopWatch class instead of DateTime.Now etc.

http://msdn.microsoft.com/en-us/library/system.diagnostics.stopwatch.aspx

Ama

Answer 4

That is quite interesting. I'm wondering if your million entry lists aren't causing a garbage collection and skewing your results. Try changing the order these functions are called in and see what the results give you.

Another thing is that the JIT might have optimised your code to not create the lambda each time and is just inserting the same value over and over. Might be worth running ildasm over it and see what is actually generated.

Wilow

Answer 5

Why is test4 and test6 much slower than their delegate version? I also noticed that Resharper specifically has a comment on the delegate version suggesting change test5 and test7 to "Covert to method group". Which is the same as test4 and test6 but they're actually slower?

You'll get a big clue by adding

        Debug.Print(ReferenceEquals(list[0], list[1]) ? "same" : "different");

to the end of each method.

With the delegate version, the Func gets compiled a bit like it was actually:

var func = Func<CacheDependency> <>_hiddenfieldwithinvalidC#name;
if (func == null)
{
  <>_hiddenfieldwithinvalidC#name = func = () => p.GetDep();
}

While with a method group it gets compiled much the same as:

func = new Func<CacheDependency>(p.GetDep());

This memoisation is done a lot with delegates created from lambdas when the compiler can determine it is safe to do so, but not with method-groups being cast to delegates, and the performance differences you see show exactly why.

I don't seem a consistent performance difference when calling test4 and test6, shouldn't static calls to be always faster?

Not necessarily. While a static call has the advantage of one less argument to pass around (as there's no implicit this argument), this difference:

1. Isn't much to begin with.
2. Might be jitted away if this isn't used.
3. Might be optimised away in that the register with the this pointer in before the call is the register with the this pointer in after the call, so there's no need to actually do anything to get it in there.
4. Eh, something else. I'm not claiming this list is exhaustive.

Really what performance benefits there are from static is more that if you do what is naturally static in instance methods you can end up with excessive passing around of objects that isn't really needed and wastes time. That said, if you are doing what is naturally instance in static methods you can end up storing/retrieving and/or allocationg and/or passing around objects in arguments you wouldn't need to and be just as bad.

Hippocrates