C# Binary Trees and Dictionaries

Question

I'm struggling with the concept of when to use binary search trees and when to use dictionaries.

In my application I did a little experiment which used the C5 library TreeDictionary (which I believe is a red-black binary search tree), and the C# dictionary. The dictionary was always faster at add/find operations and also always used less memory space. For example, at 16809 <int, float> entries, the dictionary used 342 KiB whilst the tree used 723 KiB.

I thought that BST's were supposed to be more memory efficient, but it seems that one node of the tree requires more bytes than one entry in a dictionary. What gives? Is there a point at where BST's are better than dictionaries?

Also, as a side question, does anyone know if there is a faster + more memory efficient data structure for storing <int, float> pairs for dictionary type access than either of the mentioned structures?

Maisiemaison

I honestly wouldn't worry about memory efficiency if your app is using 723 KB. I'd probably start thinking about better data structures when I hit, say, 50 MB to store the collection. – Epispastic

The object holding the data structure could have thousands of instances, so then every kB counts. – Maisiemaison

Try SortedList<T,K> - it should have the lowest memory overhead of the different options. If it's not too slow (in your application) and ever KB really does matter, it certainly seems viable. Add/remove will be slower but lookup should be similar to the BST. – Lassitude

Answer 1

I thought that BST's were supposed to be more memory efficient, but it seems that one node of the tree requires more bytes than one entry in a dictionary. What gives? Is there a point at where BST's are better than dictionaries?

I've personally never heard of such a principle. Even still, its only a general principle, not a categorical fact etched in the fabric of the universe.

Generally, Dictionaries are really just a fancy wrapper around an array of linked lists. You insert into the dictionary something like:

LinkedList<Tuple<TKey, TValue>> list =
    internalArray[internalArray % key.GetHashCode()];
if (list.Exists(x => x.Key == key))
    throw new Exception("Key already exists");
list.AddLast(Tuple.Create(key, value));

So its nearly O(1) operation. The dictionary uses O(internalArray.Length + n) memory, where n is number of items in the collection.

In general BSTs can be implemented as:

• linked-lists, which use O(n) space, where n is the number items in the collection.
• arrays, which use O(2^h - n) space where h is the height of the tree and n is the number of items in the collection.
  • Since red-black trees have a bounded height of O(1.44 * n), an array implementation should have a bounded memory usage of about O(2^1.44n - n)

Odds are, the C5 TreeDictionary is implemented using arrays, which is probably responsible for the wasted space.

What gives? Is there a point at where BST's are better than dictionaries?

Dictionaries have some undesirable properties:

• There may not be enough continugous blocks of memory to hold your dictionary, even if its memory requirements are much less than than the total available RAM.

• Evaluating the hash function can take an arbitrarily long length of time. Strings, for example, use Reflector to examine the System.String.GetHashCode method -- you'll notice hashing a string always takes O(n) time, which means it can take considerable time for very long strings. On the hand, comparing strings for inequality almost always faster than hashing, since it may require looking at just the first few chars. Its wholly possible for tree inserts to be faster than dictionary inserts if hash code evaluation takes too long.

  • Int32's GetHashCode method is literally just return this, so you'd be hardpressed to find a case where a hashtable with int keys is slower than a tree dictionary.

RB Trees have some desirable properties:

• You can find/remove the Min and Max elements in O(log n) time, compared to O(n) time using a dictionary.

• If a tree is implemented as linked list rather than an array, the tree is usually more space efficient than a dictionary.

• Likewise, its ridiculous easy to write immutable versions of trees which support insert/lookup/delete in O(log n) time. Dictionaries do not adapt well to immutability, since you need to copy the entire internal array for every operation (actually, I have seen some array-based implementations of immutable finger trees, a kind of general purpose dictionary data structure, but the implementation is very complex).

• You can traverse all the elements in a tree in sorted order in constant space and O(n) time, whereas you'd need to dump a hash table into an array and sort it to get the same effect.

So, the choice of data structure really depends on what properties you need. If you just want an unordered bag and can guarantee that your hash function evaluate quickly, go with a .Net Dictionary. If you need an ordered bag or have a slow running hash function, go with TreeDictionary.

Epispastic

"If a tree is implemented as linked list rather than an array, the tree is usually more space efficient than a dictionary." seems to be the other way around? linked list elements must also store references to accessors. – Helaine

Dictionary uses a linked list in an off-heap structure, not the LinkedList class inside. It would be 5 times slower that way. – Hallucinogen

Answer 2

It does make sense that a tree node would require more storage than a dictionary entry. A binary tree node needs to store the value and both the left and right subtrees. The generic Dictionary<TKey, TValue> is implemented as a hash table which - I'm assuming - either uses a linked list for each bucket (value plus one pointer/reference) or some sort of remapping (just the value). I'd have to have a peek in Reflector to be sure, but for the purpose of this question I don't think it's that important.

The sparser the hash table, the less efficient in terms of storage/memory. If you create a hash table (dictionary) and initialize its capacity to 1 million, and only fill it with 10,000 elements, then I'm pretty sure it would eat up a lot more memory than a BST with 10,000 nodes.

Still, I wouldn't worry about any of this if the amount of nodes/keys is only in the thousands. That's going to be measured in the kilobytes, compared to gigabytes of physical RAM.

---

If the question is "why would you want to use a binary tree instead of a hash table?" Then the best answer IMO is that binary trees are ordered whereas hash tables are not. You can only search a hash table for keys that are exactly equal to something; with a tree, you can search for a range of values, nearest value, etc. This is a pretty important distinction if you're creating an index or something similar.

Incantation

But the C# dictionary is a hashtable that automatically adjusts its size right? So by not prespecifying its size it will eventually allocate a little more than 10,000 buckets and will probably still use less memory than a tree with exactly 10,000 nodes with faster access times. Unless increasing the size of the dictionary is very slow for a large amount of elements I still don't see the advantage of trees over dictionaries. – Maisiemaison

@Projectile Fish: Generally, when you plan to populate a large dictionary, you initialize it with a specific capacity so that you don't incur the performance penalty associated with auto-growing (this is the same with almost all the generic collections). As long as your capacity estimate isn't way off, then yes, it will still likely be more memory-efficient than a tree, especially with large data sets. – Incantation

@Projectile Fish: I also added in a few lines to answer your second question, namely what would be the advantage of a tree over a dictionary. – Incantation

That makes sense. I just did a little test and it seems the auto-growing cost isn't so large. An auto-growing dictionary is still faster by 10x and more memory efficient by 2x than a TreeDictionary. So I guess the only reason to use a BST would be if I needed ordered data. – Maisiemaison

Answer 3

It seems to me you're doing a premature optimization.

What I'd suggest to you is to create an interface to isolate which structure you're actually using, and then implement the interface using the Dictionary (which seems to work best).

If memory/performance becomes an issue (which probably will not for 20k- numbers), then you can create other interface implementations, and check which one works bests. You won't need to change almost anything in the rest of the code (except which implementation you're using).

Bainmarie

Answer 4

The interface for a Tree and a Hash table (which I'm guessing is what your Dictionary is based one) should be very similar. Always revolving around keyed lookups.

I had always thought a Dictionary was better for creating things once and then then doing lots of lookups on it. While a Tree was better if you were modifying it significantly. However, I don't know where I picked that idea up from.

(Functional languages often use trees as the basis for they collections as you can re-use most of the tree if you make small modifications to it).

Relax

Answer 5

You're not comparing "apples with apples", a BST will give you an ordered representation while a dictionary allows you to do a lookup on a key value pair (in your case ).

I wouldn't expect much size in the memory footprint between the 2 but the dictionary will give you a much faster lookup. To find an item in a BST you (potentially) need to traverse the entire tree. But to do a dictnary lookup you simply lookup based on the key.

Riggs

But what is involved in "simply looking up based on the key"? With a BST, if it is relatively balanced, a lookup will be pretty quick, O(log(n)) i think? – Democracy

a lookup on a hastable would be closer to O(1) wouldn't it ? dependant on the implementation, space etc... but would defintely be quicker than a BST. – Riggs

Answer 6

A balanced BST is preferable if you need to protect your data structure from latency spikes and hash collisions attacks.

The former happens when an array-backed structure grows an gets resized, the latter is an inevitable property of hashing algorithm as a projection from infinite space to a limited integer range.

Another problem in .NET is that there is LOH, and with a sufficiently large dictionary you run into a LOH fragmentation. In this case you can use a BST, paying a price of larger algorithmic complexity class.

In short, with a BST backed by the allocation heap you get worst case O(log(N)) time, with hashtable you get O(N) worst case time.

BST comes at a price of O(log(N)) average time, worse cache locality and more heap allocations, but it has latency guarantees and is protected from dictionary attacks and memory fragmentation.

Worth noting that BST is also a subject to memory fragmentation on other platforms, not using a compacting garbage collector.

As for the memory size, the .NET Dictionary`2 class is more memory efficient, because it stores data as an off-heap linked list, which only stores value and offset information. BST has to store object header (as each node is a class instance on the heap), two pointers, and some augmented tree data for balanced trees. For example, a red-black tree would need a boolean interpreted as color (red or black). This is at least 6 machine words, if I'm not mistaken. So, each node in a red-black tree on 64-bit system is a minimum of:

3 words for the header = 24 bytes 2 words for the child pointers = 16 bytes 1 word for the color = 8 bytes at least 1 word for the value 8+ bytes = 24+16+8+8 = 56 bytes (+8 bytes if the tree uses a parent node pointer).

At the same time, the minimum size of the dictionary entry would be just 16 bytes.

Hallucinogen