What're lzo and lzf, and the differences?

Question

Hi I heard of lzo and lzf and seems they are all compression algorithms. Are they the same thing? Are there any other algorithms like them(light and fast)?

Triplet

not related to question but lrzip is quite fast and good (ratio) – Rosariarosario

Answer 1

lzo and lzf are 2 well known very simple compression algorithms. lzf goes for low memory usage during compression. lzo goes for maximum decoding speed. Both are fast, both have little memory requirements, both have comparable compression rates (which means very poor).

You can look at a direct comparison of them with other compressors here for example : http://phantasie.tonempire.net/t96-compression-benchmark#149

Serotherapy

I disagree with "very poor" as qualifier: it does 50% of what gzip/deflate, but the part included (lempel-ziv substring compression) usually yields most of compression, as the second part (huffman encoding) is slower with less impact on compression. So, lower than gzip (and bzip2) is true, but "very slow" is a subjective/relative statement. – Plumbic

Answer 2

Are there any other algorithms like them(light and fast)?

There is also LZ4 and Google's snappy. According to the benchmarks published by the LZ4 author on the project homepage and Hadoop developers on issue HADOOP-7657, LZ4 seems the fastest of them all.

Dorset

Specific benchmark suite that compares these alternatives can be found at github.com/ning/jvm-compressor-benchmark. LZ4 is indeed fastest currently. – Plumbic

Answer 3

Both are basic Lempel-Ziv compressors, which allows fast operation (since there is no second phase of encoding using huffman (as gzip/zip do) or statistical encoder) with moderate compression.

One benchmark for comparing codecs on java is jvm-compressor-benchmark. LZO is not yet included, but pure Java LZF has excellent performance (esp. compression speed), and I assume LZO might fare well too, if there was a driver for it.

Another LZ-based algorithm is Snappy by Google, and its native codec is the fastest codec at decompression (and compression is as fast as pure-java LZF compression).

Plumbic

... these days, LZ4 is faster than all of above (lzf, lzo, snappy), fwtw. – Plumbic

Answer 4

Splittable LZ4 and ZSTD for hadoop, recently born but promising -> https://github.com/carlomedas/4mc

Papism