Why is insertion into my tree faster on sorted input than random input?

Question

Now I've always heard binary search trees are faster to build from randomly selected data than ordered data, simply because ordered data requires explicit rebalancing to keep the tree height at a minimum.

Recently I implemented an immutable treap, a special kind of binary search tree which uses randomization to keep itself relatively balanced. In contrast to what I expected, I found I can consistently build a treap about 2x faster and generally better balanced from ordered data than unordered data -- and I have no idea why.

Here's my treap implementation:

• http://pastebin.com/VAfSJRwZ

And here's a test program:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Diagnostics;

namespace ConsoleApplication1
{

    class Program
    {
        static Random rnd = new Random();
        const int ITERATION_COUNT = 20;

        static void Main(string[] args)
        {
            List<double> rndTimes = new List<double>();
            List<double> orderedTimes = new List<double>();

            rndTimes.Add(TimeIt(50, RandomInsert));
            rndTimes.Add(TimeIt(100, RandomInsert));
            rndTimes.Add(TimeIt(200, RandomInsert));
            rndTimes.Add(TimeIt(400, RandomInsert));
            rndTimes.Add(TimeIt(800, RandomInsert));
            rndTimes.Add(TimeIt(1000, RandomInsert));
            rndTimes.Add(TimeIt(2000, RandomInsert));
            rndTimes.Add(TimeIt(4000, RandomInsert));
            rndTimes.Add(TimeIt(8000, RandomInsert));
            rndTimes.Add(TimeIt(16000, RandomInsert));
            rndTimes.Add(TimeIt(32000, RandomInsert));
            rndTimes.Add(TimeIt(64000, RandomInsert));
            rndTimes.Add(TimeIt(128000, RandomInsert));
            string rndTimesAsString = string.Join("\n", rndTimes.Select(x => x.ToString()).ToArray());

            orderedTimes.Add(TimeIt(50, OrderedInsert));
            orderedTimes.Add(TimeIt(100, OrderedInsert));
            orderedTimes.Add(TimeIt(200, OrderedInsert));
            orderedTimes.Add(TimeIt(400, OrderedInsert));
            orderedTimes.Add(TimeIt(800, OrderedInsert));
            orderedTimes.Add(TimeIt(1000, OrderedInsert));
            orderedTimes.Add(TimeIt(2000, OrderedInsert));
            orderedTimes.Add(TimeIt(4000, OrderedInsert));
            orderedTimes.Add(TimeIt(8000, OrderedInsert));
            orderedTimes.Add(TimeIt(16000, OrderedInsert));
            orderedTimes.Add(TimeIt(32000, OrderedInsert));
            orderedTimes.Add(TimeIt(64000, OrderedInsert));
            orderedTimes.Add(TimeIt(128000, OrderedInsert));
            string orderedTimesAsString = string.Join("\n", orderedTimes.Select(x => x.ToString()).ToArray());

            Console.WriteLine("Done");
        }

        static double TimeIt(int insertCount, Action<int> f)
        {
            Console.WriteLine("TimeIt({0}, {1})", insertCount, f.Method.Name);

            List<double> times = new List<double>();
            for (int i = 0; i < ITERATION_COUNT; i++)
            {
                Stopwatch sw = Stopwatch.StartNew();
                f(insertCount);
                sw.Stop();
                times.Add(sw.Elapsed.TotalMilliseconds);
            }

            return times.Average();
        }

        static void RandomInsert(int insertCount)
        {
            Treap<double> tree = new Treap<double>((x, y) => x.CompareTo(y));
            for (int i = 0; i < insertCount; i++)
            {
                tree = tree.Insert(rnd.NextDouble());
            }
        }

        static void OrderedInsert(int insertCount)
        {
            Treap<double> tree = new Treap<double>((x, y) => x.CompareTo(y));
            for(int i = 0; i < insertCount; i++)
            {
                tree = tree.Insert(i + rnd.NextDouble());
            }
        }
    }
}

And here's a chart comparing random and ordered insertion times in milliseconds:

Insertions         Random          Ordered         RandomTime / OrderedTime
50                 1.031665        0.261585        3.94
100                0.544345        1.377155        0.4
200                1.268320        0.734570        1.73
400                2.765555        1.639150        1.69
800                6.089700        3.558350        1.71
1000               7.855150        4.704190        1.67
2000               17.852000       12.554065       1.42
4000               40.157340       22.474445       1.79
8000               88.375430       48.364265       1.83
16000              197.524000      109.082200      1.81
32000              459.277050      238.154405      1.93
64000              1055.508875     512.020310      2.06
128000             2481.694230     1107.980425     2.24

I don't see anything in the code which makes ordered input asymptotically faster than unordered input, so I'm at a loss to explain the difference.

Why is it so much faster to build a treap from ordered input than random input?

Vidette

what might be interesting to also measure the standard deviation, so perhaps you get a good average, but the random has less fluctuation. – Blackguardly

I am interested in the comparison between treap and avl or rb-tree:) – Feudalism

It would be interesting to add counters that count how often you change structure of the tree. You can compare that with calculated values. – English

how did you test your performance using some special tools or writing it by yourself? – Purslane

This means that to build a treap from unordered input, we might process it in batches, sort each batch with a quick sorting algorithm and use treap merge instead of insert. Batches should be delimited by read accesses: so, we represent the treap by a pair (treap, unflushed inserts) and flush the inserts before each read operation. By the way, there might be explicit algorithms for building treaps from sorted input even faster. – Dealfish

If you need it to be faster during unordered input, you could make it more reluctant to rotate. It would be interesting to see a graph of the top node values, and the frequency of rotation. The optimal top level value should be 0.5. It might start off at a different value, but over time it would fluctuate a bit and probably settle around the middle. Top level rotation should be more frequent in the beginning, and less frequent later. If top level rotation happens very often over a long history, the algorithm is too "nervous". – Draughtboard

@YinZhu If you are looking for treap vs AVL vs Red-Black trees then I've done some work in Java you may be interested in. I have at least 5 tree implementations with size, add, remove, and lookup performance stats here: code.google.com/p/java-algorithms-implementation – Lally

Answer 1

Self-balancing trees exist to fix the problems associated non-randomly-distributed data. By definition, they trade away a bit of the best-case performance to vastly improve the worst-case performance associated with non-balanced BSTs, specifically that of sorted input.

You're actually overthinking this problem, because slower insertion of random data vs. ordered data is a characteristic of any balanced tree. Try it on an AVL and you'll see the same results.

Cameron had the right idea, removing the priority check to force the worst case. If you do that and instrument your tree so you can see how many rebalances are happening for each insert, it actually becomes very obvious what's going on. When inserting sorted data, the tree always rotates left and the root's right child is always empty. Insertion always results in exactly one rebalance because the insertion node has no children and no recursion occurs. On the other hand, when you run it on the random data, almost immediately you start to see multiple rebalances happening on every insert, as many as 5 or 6 of them in the smallest case (50 inserts), because it's happening on subtrees as well.

With priority checking turned back on, not only are rebalances typically less expensive due to more nodes being pushed into the left subtree (where they never come out of because no insertions happen there), but they are also less likely to occur. Why? Because in the treap, high-priority nodes float to the top, and the constant left-rotations (not accompanied by right-rotations) start to push all the high-priority nodes into the left subtree as well. The result is that rebalances happen less frequently due to the uneven distribution of probability.

If you instrument the rebalancing code you'll see that this is true; for both the sorted and random input, you end up with almost identical numbers of left-rotations, but the random input also gives the same number of right-rotations, which makes for twice as many in all. This shouldn't be surprising - Gaussian input should result in a Gaussian distribution of rotations. You'll also see that there are only about 60-70% as many top-level rebalances for the sorted input, which perhaps is surprising, and again, that's due to the sorted input messing with the natural distribution of priorities.

You can also verify this by inspecting the full tree at the end of an insertion loop. With the random input, priorities tend to decrease fairly linearly by level; with the sorted input, priorities tend to stay very high until you get to one or two levels from the bottom.

Hopefully I've done a decent job explaining this... let me know if any of it is too vague.

Sthilaire

+1, +answer: this actually makes a lot of sense, I appreciate it. And thanks to everyone else too :) – Vidette

Answer 2

I ran your code, and I think it has to do with the number of rotations. During ordered input, the number of rotations are optimal, and the tree will never have to rotate back.

During random input the tree will have to perform more rotations, because it may have to rotate back and forth.

To really find out, I would have to add counters for the numbers of left and right rotations for each run. You can probably do this yourself.

UPDATE:

I put breakpoints on rotateleft and rotateright. During ordered input rotateright is never used. During random input, both are hit, and it seems to me that they are used more frequently.

UPDATE 2:

I added some output to the 50 item ordered run (substituting with integers for clarity), to learn more:

TimeIt(50, OrderedInsert)
LastValue = 0, Top.Value = 0, Right.Count = 0, Left.Count = 0
RotateLeft @value=0
LastValue = 1, Top.Value = 1, Right.Count = 0, Left.Count = 1
LastValue = 2, Top.Value = 1, Right.Count = 1, Left.Count = 1
LastValue = 3, Top.Value = 1, Right.Count = 2, Left.Count = 1
RotateLeft @value=3
RotateLeft @value=2
RotateLeft @value=1
LastValue = 4, Top.Value = 4, Right.Count = 0, Left.Count = 4
LastValue = 5, Top.Value = 4, Right.Count = 1, Left.Count = 4
LastValue = 6, Top.Value = 4, Right.Count = 2, Left.Count = 4
RotateLeft @value=6
LastValue = 7, Top.Value = 4, Right.Count = 3, Left.Count = 4
LastValue = 8, Top.Value = 4, Right.Count = 4, Left.Count = 4
RotateLeft @value=8
RotateLeft @value=7
LastValue = 9, Top.Value = 4, Right.Count = 5, Left.Count = 4
LastValue = 10, Top.Value = 4, Right.Count = 6, Left.Count = 4
RotateLeft @value=10
RotateLeft @value=9
RotateLeft @value=5
RotateLeft @value=4
LastValue = 11, Top.Value = 11, Right.Count = 0, Left.Count = 11
LastValue = 12, Top.Value = 11, Right.Count = 1, Left.Count = 11
RotateLeft @value=12
LastValue = 13, Top.Value = 11, Right.Count = 2, Left.Count = 11
RotateLeft @value=13
LastValue = 14, Top.Value = 11, Right.Count = 3, Left.Count = 11
LastValue = 15, Top.Value = 11, Right.Count = 4, Left.Count = 11
RotateLeft @value=15
RotateLeft @value=14
LastValue = 16, Top.Value = 11, Right.Count = 5, Left.Count = 11
LastValue = 17, Top.Value = 11, Right.Count = 6, Left.Count = 11
RotateLeft @value=17
LastValue = 18, Top.Value = 11, Right.Count = 7, Left.Count = 11
LastValue = 19, Top.Value = 11, Right.Count = 8, Left.Count = 11
RotateLeft @value=19
LastValue = 20, Top.Value = 11, Right.Count = 9, Left.Count = 11
LastValue = 21, Top.Value = 11, Right.Count = 10, Left.Count = 11
RotateLeft @value=21
LastValue = 22, Top.Value = 11, Right.Count = 11, Left.Count = 11
RotateLeft @value=22
RotateLeft @value=20
RotateLeft @value=18
LastValue = 23, Top.Value = 11, Right.Count = 12, Left.Count = 11
LastValue = 24, Top.Value = 11, Right.Count = 13, Left.Count = 11
LastValue = 25, Top.Value = 11, Right.Count = 14, Left.Count = 11
RotateLeft @value=25
RotateLeft @value=24
LastValue = 26, Top.Value = 11, Right.Count = 15, Left.Count = 11
LastValue = 27, Top.Value = 11, Right.Count = 16, Left.Count = 11
RotateLeft @value=27
LastValue = 28, Top.Value = 11, Right.Count = 17, Left.Count = 11
RotateLeft @value=28
RotateLeft @value=26
RotateLeft @value=23
RotateLeft @value=16
RotateLeft @value=11
LastValue = 29, Top.Value = 29, Right.Count = 0, Left.Count = 29
LastValue = 30, Top.Value = 29, Right.Count = 1, Left.Count = 29
LastValue = 31, Top.Value = 29, Right.Count = 2, Left.Count = 29
LastValue = 32, Top.Value = 29, Right.Count = 3, Left.Count = 29
RotateLeft @value=32
RotateLeft @value=31
LastValue = 33, Top.Value = 29, Right.Count = 4, Left.Count = 29
RotateLeft @value=33
RotateLeft @value=30
LastValue = 34, Top.Value = 29, Right.Count = 5, Left.Count = 29
RotateLeft @value=34
LastValue = 35, Top.Value = 29, Right.Count = 6, Left.Count = 29
LastValue = 36, Top.Value = 29, Right.Count = 7, Left.Count = 29
LastValue = 37, Top.Value = 29, Right.Count = 8, Left.Count = 29
RotateLeft @value=37
LastValue = 38, Top.Value = 29, Right.Count = 9, Left.Count = 29
LastValue = 39, Top.Value = 29, Right.Count = 10, Left.Count = 29
RotateLeft @value=39
LastValue = 40, Top.Value = 29, Right.Count = 11, Left.Count = 29
RotateLeft @value=40
RotateLeft @value=38
RotateLeft @value=36
LastValue = 41, Top.Value = 29, Right.Count = 12, Left.Count = 29
LastValue = 42, Top.Value = 29, Right.Count = 13, Left.Count = 29
RotateLeft @value=42
LastValue = 43, Top.Value = 29, Right.Count = 14, Left.Count = 29
LastValue = 44, Top.Value = 29, Right.Count = 15, Left.Count = 29
RotateLeft @value=44
LastValue = 45, Top.Value = 29, Right.Count = 16, Left.Count = 29
LastValue = 46, Top.Value = 29, Right.Count = 17, Left.Count = 29
RotateLeft @value=46
RotateLeft @value=45
LastValue = 47, Top.Value = 29, Right.Count = 18, Left.Count = 29
LastValue = 48, Top.Value = 29, Right.Count = 19, Left.Count = 29
LastValue = 49, Top.Value = 29, Right.Count = 20, Left.Count = 29

The ordered items always gets added to the right side of the tree, naturally. When the right side gets bigger than the left, a rotateleft happens. Rotateright never happens. A new top node is selected roughly every time the tree doubles. The randomness of the priority value jitters it a little, so it goes 0, 1, 4, 11, 29 in this run.

A random run reveals something interesting:

TimeIt(50, RandomInsert)
LastValue = 0,748661640914465, Top.Value = 0,748661640914465, Right.Count = 0, Left.Count = 0
LastValue = 0,669427539533669, Top.Value = 0,748661640914465, Right.Count = 0, Left.Count = 1
RotateRight @value=0,669427539533669
LastValue = 0,318363281115127, Top.Value = 0,748661640914465, Right.Count = 0, Left.Count = 2
RotateRight @value=0,669427539533669
LastValue = 0,33133987678743, Top.Value = 0,748661640914465, Right.Count = 0, Left.Count = 3
RotateLeft @value=0,748661640914465
LastValue = 0,955126694382693, Top.Value = 0,955126694382693, Right.Count = 0, Left.Count = 4
RotateRight @value=0,669427539533669
RotateLeft @value=0,33133987678743
RotateLeft @value=0,318363281115127
RotateRight @value=0,748661640914465
RotateRight @value=0,955126694382693
LastValue = 0,641024029180884, Top.Value = 0,641024029180884, Right.Count = 3, Left.Count = 2
LastValue = 0,20709771951991, Top.Value = 0,641024029180884, Right.Count = 3, Left.Count = 3
LastValue = 0,830862050331599, Top.Value = 0,641024029180884, Right.Count = 4, Left.Count = 3
RotateRight @value=0,20709771951991
RotateRight @value=0,318363281115127
LastValue = 0,203250563798123, Top.Value = 0,641024029180884, Right.Count = 4, Left.Count = 4
RotateLeft @value=0,669427539533669
RotateRight @value=0,748661640914465
RotateRight @value=0,955126694382693
LastValue = 0,701743399585478, Top.Value = 0,641024029180884, Right.Count = 5, Left.Count = 4
RotateLeft @value=0,669427539533669
RotateRight @value=0,701743399585478
RotateLeft @value=0,641024029180884
LastValue = 0,675667521858433, Top.Value = 0,675667521858433, Right.Count = 4, Left.Count = 6
RotateLeft @value=0,33133987678743
RotateLeft @value=0,318363281115127
RotateLeft @value=0,203250563798123
LastValue = 0,531275219531392, Top.Value = 0,675667521858433, Right.Count = 4, Left.Count = 7
RotateRight @value=0,748661640914465
RotateRight @value=0,955126694382693
RotateLeft @value=0,701743399585478
LastValue = 0,704049674190604, Top.Value = 0,675667521858433, Right.Count = 5, Left.Count = 7
RotateRight @value=0,203250563798123
RotateRight @value=0,531275219531392
RotateRight @value=0,641024029180884
RotateRight @value=0,675667521858433
LastValue = 0,161392807104342, Top.Value = 0,161392807104342, Right.Count = 13, Left.Count = 0
RotateRight @value=0,203250563798123
RotateRight @value=0,531275219531392
RotateRight @value=0,641024029180884
RotateRight @value=0,675667521858433
RotateLeft @value=0,161392807104342
LastValue = 0,167598206162266, Top.Value = 0,167598206162266, Right.Count = 13, Left.Count = 1
LastValue = 0,154996359793002, Top.Value = 0,167598206162266, Right.Count = 13, Left.Count = 2
RotateLeft @value=0,33133987678743
LastValue = 0,431767346538495, Top.Value = 0,167598206162266, Right.Count = 14, Left.Count = 2
RotateRight @value=0,203250563798123
RotateRight @value=0,531275219531392
RotateRight @value=0,641024029180884
RotateRight @value=0,675667521858433
RotateLeft @value=0,167598206162266
LastValue = 0,173774613614089, Top.Value = 0,173774613614089, Right.Count = 14, Left.Count = 3
RotateRight @value=0,830862050331599
LastValue = 0,76559642412029, Top.Value = 0,173774613614089, Right.Count = 15, Left.Count = 3
RotateRight @value=0,76559642412029
RotateLeft @value=0,748661640914465
RotateRight @value=0,955126694382693
RotateLeft @value=0,704049674190604
RotateLeft @value=0,675667521858433
LastValue = 0,75742144871383, Top.Value = 0,173774613614089, Right.Count = 16, Left.Count = 3
LastValue = 0,346844367844446, Top.Value = 0,173774613614089, Right.Count = 17, Left.Count = 3
RotateRight @value=0,830862050331599
LastValue = 0,787565814232251, Top.Value = 0,173774613614089, Right.Count = 18, Left.Count = 3
LastValue = 0,734950566540915, Top.Value = 0,173774613614089, Right.Count = 19, Left.Count = 3
RotateLeft @value=0,20709771951991
RotateRight @value=0,318363281115127
RotateLeft @value=0,203250563798123
RotateRight @value=0,531275219531392
RotateRight @value=0,641024029180884
RotateRight @value=0,675667521858433
RotateRight @value=0,75742144871383
RotateLeft @value=0,173774613614089
LastValue = 0,236504829598826, Top.Value = 0,236504829598826, Right.Count = 17, Left.Count = 6
RotateLeft @value=0,830862050331599
RotateLeft @value=0,787565814232251
RotateLeft @value=0,76559642412029
RotateRight @value=0,955126694382693
LastValue = 0,895606500048007, Top.Value = 0,236504829598826, Right.Count = 18, Left.Count = 6
LastValue = 0,599106418713511, Top.Value = 0,236504829598826, Right.Count = 19, Left.Count = 6
LastValue = 0,8182332901369, Top.Value = 0,236504829598826, Right.Count = 20, Left.Count = 6
RotateRight @value=0,734950566540915
LastValue = 0,704216948572647, Top.Value = 0,236504829598826, Right.Count = 21, Left.Count = 6
RotateLeft @value=0,346844367844446
RotateLeft @value=0,33133987678743
RotateRight @value=0,431767346538495
RotateLeft @value=0,318363281115127
RotateRight @value=0,531275219531392
RotateRight @value=0,641024029180884
RotateRight @value=0,675667521858433
RotateRight @value=0,75742144871383
LastValue = 0,379157059536854, Top.Value = 0,236504829598826, Right.Count = 22, Left.Count = 6
RotateLeft @value=0,431767346538495
LastValue = 0,46832062046431, Top.Value = 0,236504829598826, Right.Count = 23, Left.Count = 6
RotateRight @value=0,154996359793002
LastValue = 0,0999000217299443, Top.Value = 0,236504829598826, Right.Count = 23, Left.Count = 7
RotateLeft @value=0,20709771951991
LastValue = 0,229543754006524, Top.Value = 0,236504829598826, Right.Count = 23, Left.Count = 8
RotateRight @value=0,8182332901369
LastValue = 0,80358425984326, Top.Value = 0,236504829598826, Right.Count = 24, Left.Count = 8
RotateRight @value=0,318363281115127
LastValue = 0,259324726769386, Top.Value = 0,236504829598826, Right.Count = 25, Left.Count = 8
RotateRight @value=0,318363281115127
LastValue = 0,307835293145774, Top.Value = 0,236504829598826, Right.Count = 26, Left.Count = 8
RotateLeft @value=0,431767346538495
LastValue = 0,453910283024381, Top.Value = 0,236504829598826, Right.Count = 27, Left.Count = 8
RotateLeft @value=0,830862050331599
LastValue = 0,868997387527021, Top.Value = 0,236504829598826, Right.Count = 28, Left.Count = 8
RotateLeft @value=0,20709771951991
RotateRight @value=0,229543754006524
RotateLeft @value=0,203250563798123
LastValue = 0,218358597354199, Top.Value = 0,236504829598826, Right.Count = 28, Left.Count = 9
RotateRight @value=0,0999000217299443
RotateRight @value=0,161392807104342
LastValue = 0,0642934488431986, Top.Value = 0,236504829598826, Right.Count = 28, Left.Count = 10
RotateRight @value=0,154996359793002
RotateLeft @value=0,0999000217299443
LastValue = 0,148295871982489, Top.Value = 0,236504829598826, Right.Count = 28, Left.Count = 11
LastValue = 0,217621828065078, Top.Value = 0,236504829598826, Right.Count = 28, Left.Count = 12
RotateRight @value=0,599106418713511
LastValue = 0,553135806020878, Top.Value = 0,236504829598826, Right.Count = 29, Left.Count = 12
LastValue = 0,982277666210326, Top.Value = 0,236504829598826, Right.Count = 30, Left.Count = 12
RotateRight @value=0,8182332901369
LastValue = 0,803671114520948, Top.Value = 0,236504829598826, Right.Count = 31, Left.Count = 12
RotateRight @value=0,203250563798123
RotateRight @value=0,218358597354199
LastValue = 0,19310415405459, Top.Value = 0,236504829598826, Right.Count = 31, Left.Count = 13
LastValue = 0,0133136604043253, Top.Value = 0,236504829598826, Right.Count = 31, Left.Count = 14
RotateLeft @value=0,46832062046431
RotateRight @value=0,531275219531392
RotateRight @value=0,641024029180884
RotateRight @value=0,675667521858433
RotateRight @value=0,75742144871383
LastValue = 0,483394719419719, Top.Value = 0,236504829598826, Right.Count = 32, Left.Count = 14
RotateLeft @value=0,431767346538495
RotateRight @value=0,453910283024381
LastValue = 0,453370328738061, Top.Value = 0,236504829598826, Right.Count = 33, Left.Count = 14
LastValue = 0,762330518459124, Top.Value = 0,236504829598826, Right.Count = 34, Left.Count = 14
LastValue = 0,699010426969738, Top.Value = 0,236504829598826, Right.Count = 35, Left.Count = 14

Rotations happen not so much because the tree is unbalanced, but because of the priorities, which are randomly selected. For example we get 4 rotations at the 13th insertion. We have a tree balanced at 5/7 (which is fine), but get to 13/0! It would seem that the use of random priorities deserves further investigation. Anyhow, it is plain to see that the random inserts cause a lot more rotations, than the ordered inserts.

Draughtboard

Left and right rotations are symmetrical. It should not matter if you do 5-0 or 2-3. Unless you really do more of them in one case vs the other. Is this the case? – Musgrove

You are right. The sorted case does half as many rotations on average than the random - that makes a big difference. +1 – Musgrove

The height of the tree isn't used for balancing, only the priority, and every node has a fixed priority. Once the tree is rotated for priority, I don't expect (and in fact I won't ever) rotate the tree in a way which violates the heap property. When I insert a node into the tree, I expect its calculated priority to be on average greater than at least 50% of the elements in the tree, and 50% of the elements along the path of insertion. Its still not obvious to me why the tree would have twice as many rotations with random data than ordered data. – Vidette

@Juliet: that's right. However the number of rotations is bounded by the length of the insertion path and not the height of the tree. – Musgrove

Answer 3

I added calculation of the standard deviation, and changed your test to run at the highest priority (to reduce noise as much as possible). This are the results:

Random                                   Ordered
0,2835 (stddev 0,9946)                   0,0891 (stddev 0,2372)
0,1230 (stddev 0,0086)                   0,0780 (stddev 0,0031)
0,2498 (stddev 0,0662)                   0,1694 (stddev 0,0145)
0,5136 (stddev 0,0441)                   0,3550 (stddev 0,0658)
1,1704 (stddev 0,1072)                   0,6632 (stddev 0,0856)
1,4672 (stddev 0,1090)                   0,8343 (stddev 0,1047)
3,3330 (stddev 0,2041)                   1,9272 (stddev 0,3456)
7,9822 (stddev 0,3906)                   3,7871 (stddev 0,1459)
18,4300 (stddev 0,6112)                  10,3233 (stddev 2,0247)
44,9500 (stddev 2,2935)                  22,3870 (stddev 1,7157)
110,5275 (stddev 3,7129)                 49,4085 (stddev 2,9595)
275,4345 (stddev 10,7154)                107,8442 (stddev 8,6200)
667,7310 (stddev 20,0729)                242,9779 (stddev 14,4033)

I've ran a sampling profiler and here are the results (amount of times the program was in this method):

Method           Random        Ordered
HeapifyRight()   1.95          5.33
get_IsEmpty()    3.16          5.49
Make()           3.28          4.92
Insert()         16.01         14.45
HeapifyLeft()    2.20          0.00

Conclusion: the random has a fairly reasonable distribution between left and right rotation, while the ordered never rotates left.

Here is my improved "benchmark" program:

    static void Main(string[] args)
    {
        Thread.CurrentThread.Priority = ThreadPriority.Highest;
        Process.GetCurrentProcess().PriorityClass = ProcessPriorityClass.RealTime;

        List<String> rndTimes = new List<String>();
        List<String> orderedTimes = new List<String>();

        rndTimes.Add(TimeIt(50, RandomInsert));
        rndTimes.Add(TimeIt(100, RandomInsert));
        rndTimes.Add(TimeIt(200, RandomInsert));
        rndTimes.Add(TimeIt(400, RandomInsert));
        rndTimes.Add(TimeIt(800, RandomInsert));
        rndTimes.Add(TimeIt(1000, RandomInsert));
        rndTimes.Add(TimeIt(2000, RandomInsert));
        rndTimes.Add(TimeIt(4000, RandomInsert));
        rndTimes.Add(TimeIt(8000, RandomInsert));
        rndTimes.Add(TimeIt(16000, RandomInsert));
        rndTimes.Add(TimeIt(32000, RandomInsert));
        rndTimes.Add(TimeIt(64000, RandomInsert));
        rndTimes.Add(TimeIt(128000, RandomInsert));
        orderedTimes.Add(TimeIt(50, OrderedInsert));
        orderedTimes.Add(TimeIt(100, OrderedInsert));
        orderedTimes.Add(TimeIt(200, OrderedInsert));
        orderedTimes.Add(TimeIt(400, OrderedInsert));
        orderedTimes.Add(TimeIt(800, OrderedInsert));
        orderedTimes.Add(TimeIt(1000, OrderedInsert));
        orderedTimes.Add(TimeIt(2000, OrderedInsert));
        orderedTimes.Add(TimeIt(4000, OrderedInsert));
        orderedTimes.Add(TimeIt(8000, OrderedInsert));
        orderedTimes.Add(TimeIt(16000, OrderedInsert));
        orderedTimes.Add(TimeIt(32000, OrderedInsert));
        orderedTimes.Add(TimeIt(64000, OrderedInsert));
        orderedTimes.Add(TimeIt(128000, OrderedInsert));
        var result = string.Join("\n", (from s in rndTimes
                        join s2 in orderedTimes
                            on rndTimes.IndexOf(s) equals orderedTimes.IndexOf(s2)
                        select String.Format("{0} \t\t {1}", s, s2)).ToArray());
        Console.WriteLine(result);
        Console.WriteLine("Done");
        Console.ReadLine();
    }

    static double StandardDeviation(List<double> doubleList)
    {
        double average = doubleList.Average();
        double sumOfDerivation = 0;
        foreach (double value in doubleList)
        {
            sumOfDerivation += (value) * (value);
        }
        double sumOfDerivationAverage = sumOfDerivation / doubleList.Count;
        return Math.Sqrt(sumOfDerivationAverage - (average * average));
    }
    static String TimeIt(int insertCount, Action<int> f)
    {
        Console.WriteLine("TimeIt({0}, {1})", insertCount, f.Method.Name);

        List<double> times = new List<double>();
        for (int i = 0; i < ITERATION_COUNT; i++)
        {
            Stopwatch sw = Stopwatch.StartNew();
            f(insertCount);
            sw.Stop();
            times.Add(sw.Elapsed.TotalMilliseconds);
        }

        return String.Format("{0:f4} (stddev {1:f4})", times.Average(), StandardDeviation(times));
    }

Blackguardly

Answer 4

Yes it's the number of rotations that is causing the extra time. Here's what I did:

• Remove the lines checking priority in HeapifyLeft and HeapifyRight so rotations are always done.
• Added a Console.WriteLine after the if in RotateLeft and RotateRight.
• Added a Console.WriteLine in the IsEmpty part of the Insert method to see what was being inserted.
• Ran the test once with 5 values each.

Output:

TimeIt(5, RandomInsert)
Inserting 0.593302943554382
Inserting 0.348900582338171
RotateRight
Inserting 0.75496212381635
RotateLeft
RotateLeft
Inserting 0.438848891499848
RotateRight
RotateLeft
RotateRight
Inserting 0.357057290783644
RotateLeft
RotateRight

TimeIt(5, OrderedInsert)
Inserting 0.150707998383189
Inserting 1.58281302712057
RotateLeft
Inserting 2.23192588297274
RotateLeft
Inserting 3.30518679009061
RotateLeft
Inserting 4.32788012657682
RotateLeft

Result: 2 times as many rotations on random data.

Gelatinoid

If you remove the check, what are you measuring? – Musgrove

Removing the checks in HeapifyLeft/Right forces the algorithm to always do the worst case - perform the rotation. – Gelatinoid

Measuring 5 inserts will likely have a high signal to noise ratio, give it a try with my extended benchmark program? – Blackguardly

@Cameron, @Davy: I've measured without modifying the code and with 16000 iterations. The results are the same: in the random case, there are 2x as many rotations done. – Musgrove

@Davy I'm not saying it's slower due to the time measured over 5 inserts, I'm saying over 5 inserts Random does twice as many rotates as Ordered. @Andras It wasn't a guess :) – Gelatinoid

Answer 5

You're only seeing a difference of about 2x. Unless you've tuned the daylights out of this code, that's basically in the noise. Most well-written programs, especially those involving data structure, can easily have more room for improvement than that. Here's an example.

I just ran your code and took a few stackshots. Here's what I saw:

Random Insert:

1 Insert:64 -> HeapifyLeft:81 -> RotateRight:150
1 Insert:64 -> Make:43 ->Treap:35
1 Insert:68 -> Make:43

Ordered Insert:

1 Insert:61
1 OrderedInsert:224
1 Insert:68 -> Make:43
1 Insert:68 -> HeapifyRight:90 -> RotateLeft:107
1 Insert:68
1 Insert:68 -> Insert:55 -> IsEmpty.get:51

This is a pretty small number of samples, but it suggests in the case of random input that Make (line 43) is consuming a higher fraction of time. That is this code:

    private Treap<T> Make(Treap<T> left, T value, Treap<T> right, int priority)
    {
        return new Treap<T>(Comparer, left, value, right, priority);
    }

I then took 20 stackshots of the Random Insert code to get a better idea of what it was doing:

1 Insert:61
4 Insert:64
3 Insert:68
2 Insert:68 -> Make:43
1 Insert:64 -> Make:43
1 Insert:68 -> Insert:57 -> Make:48 -> Make:43
2 Insert:68 -> Insert:55
1 Insert:64 -> Insert:55
1 Insert:64 -> HeapifyLeft:81 -> RotateRight:150
1 Insert:64 -> Make:43 -> Treap:35
1 Insert:68 -> HeapifyRight:90 -> RotateLeft:107 -> IsEmpty.get:51
1 Insert:68 -> HeapifyRight:88
1 Insert:61 -> AnonymousMethod:214

This reveals some information.
25% of time is spent in line Make:43 or its callees.
15% of time is spent in that line, not in a recognized routine, in other words, in new making a new node.
90% of time is spent in lines Insert:64 and 68 (which call Make and heapify.
10% of time is spent in RotateLeft and Right.
15% of time is spent in Heapify or its callees.

I also did a fair amount of single-stepping (at the source level), and came to the suspicion that, since the tree is immutable, it spends a lot of time making new nodes because it doesn't want to change old ones. Then the old ones are garbage collected because nobody refers to them anymore.

This has got to be inefficient.

I'm still not answering your question of why inserting ordered numbers is faster than randomly generated numbers, but it doesn't really surprise me, because the tree is immutable.

I don't think you can expect any performance reasoning about tree algorithms to carry over easily to immutable trees, because the slightest change deep in the tree causes it to be rebuilt on the way back out, at a high cost in new-ing and garbage collection.

Posthaste

Answer 6

@Guge is right. However there is a little tiny bit more to it. I am not saying that it is the biggest factor in this case - however it is there and it is hard to do anything about it.

For a sorted input, lookups likely touch the nodes that are hot in the cache. (This is true in general for balanced trees like AVL trees, red-black trees, B-trees, etc.)

Since inserts start with a lookup, this has an effect on insert/delete performance as well.

Again, I am not claiming that it is the most significant factor in every and all cases. It is there, however, and will likely result in sorted inputs being always faster than random ones in these data structures.

Musgrove

If you add sorted inputs to a tree that doesn't balance itself, the tree becomes a linked list. The only question then is if you add the new nodes to the bottom of the tree, or build a new top out of the new node and add the existing tree to it. The latter is of course O(1) while the former is O(n). – Draughtboard

Since the tree is immutable, everytime you add a new node, you create brand new nodes along the insertion path. Would there still be cache affect if I'm always creating new nodes? – Vidette

@Juliet: since in the next round you are likely to visit the newly created nodes, I would guess, yes - though you might not benefit from sorted input as much as a mutable treap would. However if you do a benchmark with lookups only on an existing treap, you will most likely see a noticeable difference between random and sorted lookups. – Musgrove

Answer 7

Aaronaught has done a really decent job explaining this.

For these two special cases, I find it easier to grasp it in terms of the insertion path lengths.

For random input, your insertion path goes down to one of the leaves and the length of the path - thus the number of rotations - are bounded by the height of the tree.

In the sorted case, you walk on the right spine of the treap and the bound is the length of the spine, which is less than or equal to the the height.

Since you rotate nodes along the insertion path and your insertion path is the spine in this case, these rotations will always shorten the spine (which will result in a shorter insertion path at the next insertion, since the insertion path is just the spine etc.)

Edit: for the random case the insertion path is 1.75x longer.

Musgrove

This is also a very important point. One of the things I remember from digging around yesterday was that the height of the spine is much shorter than the height of the entire tree - a tree with height of 30 might have a spine as short as 15 or even 10 nodes. So this is actually a triple-whammy for performance - rebalances are less frequent and less expensive and the lookup time for each insert is smaller! I'm not sure which factor is the dominant one, but together they definitely make a major impact. – Sthilaire

@Aaronaught: @Vidette couldn't have found a more complicated data structure to analyze. :-) This beast has three in one: properties of binary trees, properties of binary heaps and the whole is stirred up by some probability theory (this is without analyzing run-time characteristics on real-world hardware). I bet they torture CS students with it. Uh, oh. Yes, they do: ocw.mit.edu/NR/rdonlyres/… xD – Musgrove

@Aaronaught: Your question about the dominant factor really bugged me, so I updated the answer. ;-) – Musgrove

@Aaronaught: The thing is, this treap is immutable, so the time includes the cost of rebuilding on the way out after every alteration. Wouldn't that have to be factored into all reasoning about performance? – Posthaste

@Mike Dunlavey: The immutability definitely makes it more expensive than the equivalent mutable structure, but it should be a constant overhead - shouldn't have any significant impact on sorted vs. random insertions because it's the same number of nodes either way. – Sthilaire

@andras: Funny stuff, I bet many of the CS students have never had to learn probability theory so that would be torture... then again, that question is from MIT. :P – Sthilaire

Answer 8

Try this: database on treap.

http://code.google.com/p/treapdb/

Ilene