What does "nodetool compact" do for DateTieredCompactionStrategy?

Question

The documentation for nodetool compact says:

This command starts the compaction process on tables that use the SizeTieredCompactionStrategy and DateTieredCompactionStrategy. You can specify a keyspace for compaction.

But what it does for DateTieredCompactionStrategy?

Side question: what is -s, --split-output parameter? It is explained as: Use -s to not create a single big file. I'm confused - isn't that the purpose of nodetool compact?

Stinking

Answer 1

Nodetool compact with no flags will still create a big single file even with DTCS.

The -s, --split-output option is only there starting with c* 2.2 and beyond.

The news.txt states:

 +     It is also possible to split output when doing a major compaction with
 +     STCS - files will be split in sizes 50%, 25%, 12.5% etc of the total size.
 +     This might be a bit better than old major compactions which created one big
 +     file on disk.

On DTCS -s won't do anything special (will still create one large sstable)

Constable

Many thanks for the answer! Side question: any clue what can I do when for one or two days the compaction was broken (until the node restart) and during that time the node created 2000 SSTables without compacting them? – Stinking

How was it broken? Did you turn off autocompaction? – Constable

I'm not sure, but there are two options: 1. nodetool repair failed and it didn't clear/compact created 2000 sstables, OR 2. after nodetool repair failed, a bug stopped autocompaction until restart. – Stinking

More likely your repair just generated lots of sstables. Autocompaction is still running but is a bit behind. Are you running vnodes? – Constable

Yes, that's also possible. The problem is that the age of generated sstables passed max_sstable_age_days. Therefore, those many sstables won't be automatically compacted. Yes, I'm using vnodes. – Stinking

This is actually a known limitation of older versions of DTCS, due to max_sstable_age_days. I discussed it in detail at Cassandra Summit 2015 ( slideshare.net/JeffJirsa1/… ) - you'll need to either increase max_sstable_age_days OR manually join the tiny sstables (which came from repair) using forceUserDefinedCompaction. Or you could try github.com/jeffjirsa/twcs , which I wrote to address this limitation. – Placeman