Small sized collections from C5 Generic Collection Library are comparatively very slow - can anything be done?

Question

I've recently been testing C5 collections in C# and I'm loving their functionality. For large collections the performance seems on par with the generic counterparts. For small collections however they are significantly slower. I suspect the dramatic deterioration in relative speed comes from constant time operations performed by C5 collections. One operation I know of is firing of events. Could this be the cause of poor performance for small collections? Can this perhaps be remedied by turning some functionality off? Here' the performance test:

//Two containers to be tested. 'Test' is a wrapper over decimal.
var arrayList = new C5.ArrayList<Test>();
var genericList = new System.Collections.Generic.List<Test>();

var toBeAdded = new List<Test>();
var watch = new Stopwatch();

//Fill both tested containers
for (int i = 10; i > 0; i--)
{
    var test = new Test(i);
    genericList.Add(test);
    arrayList.Add(test);
}

//Fill the container the range of which will be inserted to the tested containers
for (int i = 5; i > 0; i--)
{
    toBeAdded.Add(new Test(i+0.5m));
}


//Test the speed of adding a range of values and sorting for both containers
watch.Start();
genericList.AddRange(toBeAdded);
Console.WriteLine("Adding values for generic list: {0} ticks", watch.ElapsedTicks);
watch.Restart();
genericList.Sort();
Console.WriteLine("Sorting for generic list: {0} ticks", watch.ElapsedTicks);

watch.Restart();
arrayList.AddAll(toBeAdded);
Console.WriteLine("Adding values for C5 ArrayList: {0} ticks", watch.ElapsedTicks);
watch.Restart();
arrayList.Sort();
Console.WriteLine("Sorting for C5 ArrayList: {0} ticks", watch.ElapsedTicks);

and the Test class:

class Test : IComparable
{
    private decimal _number;
    internal Test(decimal aNumber)
    {
        _number = aNumber;
    }        
    public int CompareTo(object obj)
    {
        var test = (Test) obj;
        return _number.CompareTo(test._number);
    } 
}

The output is:

Adding values for generic list: 56 ticks
Sorting for generic list: 770 ticks
Adding values for C5 ArrayList: 3575 ticks
Sorting for C5 ArrayList: 4815 ticks

Both C5 and the test are Release builds. The ratio of speeds of around 60x for inserting and 6x for sorting is consistent between test runs.

EDIT: The above test was ran from within VS. The results for running outside of VS are:

Adding values for generic list: 54 ticks
Sorting for generic list: 2135 ticks
Adding values for C5 ArrayList: 5765 ticks
Sorting for C5 ArrayList: 5198 ticks

Again, the ratio of speeds of around 100x for inserting and 2x for sorting is consistent between test runs.

My project includes a lot of manipulating of small containers and their performance is paramount. The functionality of C5 containers is great and I'd love to use them, but at the moment can't for performance reasons. I'd appreciate any insight on the matter.

EDIT2: As per Iridium answer, I performed the test in a loop (putting the whole logic including the container creation into the the loop so as to exclude any compiler optimization tricks), discarded the first two results and averaged subsequent 1000 results. Here they are:

Adding values for generic list: 1.09 ticks
Sorting for generic list: 14.07 ticks
Adding values for C5 ArrayList: 1.92 ticks
Sorting for C5 ArrayList: 13.69 ticks

Now C5 insertion is 76% slower and sorting is on par with that of List. That's good enough for my purpose. I'm accepting Iridium's answer. Still, if anyone has any insight on the slower insertion, please share it. Thanks all for the help.

Winegrower

Just to be sure, you're running release build a couple of times outside VS etc? – Luiseluiza

@Luiseluiza I'm running Release builds of both the test and C5 dll itself. I'm running every test a few times and the results are consistent. The above was ran inside VS; I've added results for an outside-of-VS run. They're a little different but the performance issue still remains. Thanks for the question. – Winegrower

Having a brief look at the C5 ArrayList in a decompiler I would think that the answer is "no, you can't improve performance". C5 has a more layered approach compared to a normal List and apparently this has a cost in terms of performance. – Apocalyptic

@MartinLiversage This is my impression also, not without a ground-up modifying of C5. Pity, it seems I'll have to extend List<T> with some of C5's useful methods. (If you're using Reflector on the dll, the well-commented source can be got here). Thanks for looking into this. – Winegrower

Answer 1

I'm wondering here if your results are a little misleading, and that the differences you're seeing for C5 are in fact (possibly) due to the overhead of assembly loading/JIT compilation on the first use of the add/sort method. The generic collections not suffering from this as a result of the system assemblies having already been already loaded/NGen'ed.

I repeated your test using C5 2.1.4596.30350, but ran the whole test multiple times (without restarting the application so as to avoid any one-off overheads). The results seem to suggest that there is an time penalty on first use of C5 collections (consistent with a one-off overhead like JIT compilation) that disappears on subsequent uses, leaving C5 performance effectively the same as that of the generic collections.

Schizophyceous

Thanks for your answer. I'm getting very similar results and they're good enough for me to use C5 (it's difficult to tell the difference between adds in your test because you use logarithmic scale). Out of interest, did you plug the results of testing into a third party plotting app, or are you using some integrated suite for performance testing? – Winegrower

@Winegrower I tweaked your Console.WriteLine()s so that they output CSV formatted data, and dropped them into Excel (2007) to produce the chart, nothing fancy. – Schizophyceous