Why is it called rainbow table?

Question

Anyone know why it is called rainbow table? Just remembered we have learned there is an attack called "dictionary attack". Why it is not call dictionary?

Luthuli

If you've an interest in crypto I recommend supporting area51.stackexchange.com/proposals/15811/cryptography – Admittance

FYI, You accepted an answer that is invalid. Check resonances's answer for example. Also, in the original paper, it is said: ( lasec.epfl.ch/pub/lasec/doc/Oech03.pdf ) We call our chains rainbow chains. They use a successive reduction function for each point in the chain. And that's about it, they call it Rainbow tables because they use a different reduction function on each column on the table – Bubbly

Answer 1

Because it contains the entire "spectrum" of possibilities.

A dictionary attack is a bruteforce technique of just trying possibilities. Like this (python pseudo code)

mypassworddict = dict()

for password in mypassworddict:
    trypassword(password)

However, a rainbow table works differently, because it's for inverting hashes. A high level overview of a hash is that it has a number of bins:

bin1, bin2, bin3, bin4, bin5, ...

Which correspond to binary parts of the output string - that's how the string ends up the length it is. As the hash proceeds, it affects differing parts of the bins in different ways. So the first byte (or whatever input field is accepted) input affects (say, simplistically) bins 3 and 4. The next input affects 2 and 6. And so on.

A rainbow table is a computation of all the possibilities of a given bin, i.e. all the possible inverses of that bin, for every bin... that's why it ends up so large. If the first bin value is 0x1 then you need to have a lookup list of all the values of bin2 and all the values of bin3 working backwards through the hash, which eventually gives you a value.

Why isn't it called a dictionary attack? Because it isn't.

As I've seen your previous question, let me expand on the detail you're looking for there. A cryptographically secure hash needs to be safe ideally from smallish input sizes up to whole files. To precompute the values of a hash for an entire file would take forever. So a rainbow table is designed on a small well understood subset of outputs, for example the permutations of all the characters a-z over a field of say 10 characters.

This is why password advice for defeating dictionary attacks works here. The more subsets of the whole possible set of inputs you put into your input for the hash, the more a rainbow table needs to contain to search it. The data sizes required end up stupidly big and so does the time to search. So, think about it:

• If you have an input that is [a-z] for 5-8 characters, that's not too bad a rainbow table.
• If you increase the length to 42 characters, that's a massive rainbow table. Each input affects the hash and so the bins of said hash.
• If you throw numbers in to your search requirement [a-z][0-9] you've got even more searching to do.
• Likewise [A-Za-z0-9]. Finally, stick in [\w] i.e. any printable character you can think of, and again, you're looking at a massive table.

So, making passwords long and complicated makes rainbow tables start taking blue-ray sized discs of data. Then, as per your previous question, you start adding in salting and hash derived functions and you make a general solution to hash cracking hard(er).

The goal here is to stay ahead of the computational power available.

Admittance

there are subclass of dictionary attacks which uses a precomputed dictionary. – Chromaticness

Answer 2

Rainbow is a variant of dictionary attack (Pre-computed dictionary attack to be exact), but it takes less space than full dictionary (at the price of time needed to find a key in table). The other end of this space-memory tradeoff is full search (brute force attack = zero precomputation, a lot of time).

In the rainbow table the precomputed dictionary of pairs key-ciphertext is compressed in chains. Every step in chain is done using different commpression function. And the table has a lot of chains, so it looks like a rainbow.

In this picture different compression functions K1, K2, K3 have a colors like in rainbow: The table, stored in the file contains only first and last columns, as the middle columns can be recomputed.

Chromaticness

Interesting. +1. I have a high level understanding of how things work; I must read into it in more detail I think! – Admittance

@user496949, if you need an additional explanation, just ask. I read several papers about it and understand principle of rainbow table – Chromaticness

Answer 3

I don't know where the name comes from, but the differences are:

• A dictionary contains a few selected items (e.g. english words), while a rainbow table contains every possible combination.
• A dictionary only contains the input, while the rainbow table contains both the input and the output.
• A dictionary is used to test different input to see if the output is valid, while a rainbow table is used for e reverse lookup, i.e. to find which input gives a specific output.

Tincture

If u dont know, you can try wikipedia: en.wikipedia.org/wiki/Dictionary_attack This is about Pre-computed dictionary attack – Chromaticness

Not exactly true that dictionary contains few items and rainbow table contains all combinations. It all depends on what key-value or rainbow chains you put into dictionary and rainbow table respectively. Rainbow chains are simply more space efficient. – Heraldic

rainbow table can also contain a part of search space, e.g. all alpha-numberic of length i, or all numberic of length 2i and so on – Chromaticness

@osgx: I don't see what you mean. I checked out the page that you linked to, and I can't find anything about where the name came from. – Tincture

Guffa, there can be variant of dictionary which has both inputs and outputs (precomputed dictionary) – Chromaticness

@osgx: That didn't make it any clearer at all. What do you mean? Is there anything on the page that you linked to that has anything to do with where the name comes from? – Tincture

Answer 4

Unfortunately some of the statements are not correct. Contrary to what is bring posted rainbow tables DO NOT contain all the possibilites for a given keyspace well not the ones generated for use that I've seen. They can be generated to cover 99.9 but due to the randomness of a hash function there in no gurantee that EVERY plaintext is covered.

Each chain is made up of links or steps and each step is made of a hashing and reduction function. If your chain was 100 links long you would go that number of hash/reduction functions then discarding everything in between except the start and end.

To find the plain for a given hash you simply perform the reduction / hash x amount of the length of your chain. So you run the step once and check against the endpoint if it's a miss you would repeat... Until you have stepped through the entire length of your chain. If there is a match you can then regenerate the chain from the start point and you may be able to find the plain. If after the regeneration it is not correct then this is a false alarm. This happens due to collisions caused by the reduction hashing function. Since the table contains many chains you can do a large lookup against all the chain endpoints each step, this is essentially where the magic happens allowing speed. This will also lead to false alarms, since you only need to regenerate chains which have matches you save lots of time by skipping unnecessary chains.

They do not contain dictionaries.... Well not the traditional tables there are variants of rainbow tables which incorporate the use of dictionaries though.

That's about it. There are many ways which this process has been optimized including removing merging / duplicate chains and creating perfect tables and also storing them in differing packing to save space and loading time.

Wilmington