Memcache maximum key expiration time

Question

What's memcached's maximum key expiration time?

If I don't provide an expiration time and the cache gets full, what happens?

Piety

Answer 1

You can set key expiration to a date, by supplying a Unix timestamp instead of a number of days. This date can be more than 30 days in the future:

Expiration times are specified in unsigned integer seconds. They can be set from 0, meaning "never expire", to 30 days (60*60*24*30). Any time higher than 30 days is interpreted as a unix timestamp date. If you want to expire an object on january 1st of next year, this is how you do that.

https://github.com/memcached/memcached/wiki/Programming#expiration

But, as you say, if you’re setting key expiration to an amount of time rather than a date, the maximum is 2,592,000 seconds, or 30 days.

Milkfish

OMG, We had numerous cases in Rails of :expires_in => 1.month. Change to 1.month.from_now to use a date. – Baggage

The link referenced is dead :( – Bryon

@PaulD.Waite Great reactivity! I read the article and I'm still unsure about the behavior of everything. If you have a couple of minutes to spare and can take a look at the specific of my question: #22501324 , it would be most appreciated :) – Bryon

@marcgg: well it helps we’re in similar time zones. Ah yes I see — I’m not familiar enough with memcached to know the answer to your question, but it’s definitely well-written, so I’ve voted it up and tweaked the title. Hopefully that will help attract some attention. – Milkfish

@marcgg: note that @Eimantas’ answer below suggests that the oldest keys will be expired first in the situation you describe. – Milkfish

Link dead again – Atli

Answer 2

If you don't provide expiration and cache gets full then the oldest key-values are expired first:

Memory is also reclaimed when it's time to store a new item. If there are no free chunks, and no free pages in the appropriate slab class, memcached will look at the end of the LRU for an item to "reclaim". It will search the last few items in the tail for one which has already been expired, and is thus free for reuse. If it cannot find an expired item however, it will "evict" one which has not yet expired. This is then noted in several statistical counters

https://github.com/memcached/memcached/wiki/UserInternals#when-are-items-evicted

Garbo

are you 100% sure about this? – Recurrence

Yes, @DeeJay', just found this article: code.google.com/p/memcached/wiki/NewUserInternals – Idiotic

Memory is also reclaimed when it's time to store a new item. If there are no free chunks, and no free pages in the appropriate slab class, memcached will look at the end of the LRU for an item to "reclaim". It will search the last few items in the tail for one which has already been expired, and is thus free for reuse. If it cannot find an expired item however, it will "evict" one which has not yet expired. This is then noted in several statistical counters. – Idiotic

Where do you see in the stats the default expiration time? – Entozoon

@NathanB i think the stats mentioned above is for eviction. For example: STAT evictions 0 There is no default expiration time. Its set per key basis and no stat is available for this. – Baltic

Answer 3

No there is no limit. The 30 days limit is if you give the amount of seconds it should stay there, but if you give a timestamp, there is only the max long or int value on the machine which can be a limit.

->set('key', 'value', time() + 24*60*60*365) will make the key stay there for a year for example, but yeah if the cache gets full or restarted in between, this value can be deleted.

Taenia

Answer 4

An expiration time, in seconds. Can be up to 30 days. After 30 days, is treated as a unix timestamp of an exact date.

https://code.google.com/p/memcached/wiki/NewCommands#Standard_Protocol

Marciamarciano

I just want to point out how surprising this behavior is. A developer might think they're specifying a TTL duration larger than 30 days. But because Memcached treats values >2592000 as Unix timestamps, a "TTL" of (say) 60 days will get interpreted as a timestamp from the year 1970, meaning the key immediately expires! It's effectively a silent failure mode. – Folie

Answer 5

OK, I found out that the number of seconds may not exceed 2592000 (30 days). So the maximum expiration time is 30 days.

Piety

No, that's not true. The number of seconds in a duration form may not exceed 30 days. You can specify an expiration three months out if you want to as an absolute time. You can also specify 0 to not have an expiration. – Buddleia

I've also found out that if you provide too long expiration time the value is not stored at all. – Sapphism

I've found that 30 days doesn't work (doesn't write cache) but 29 days does work. – Pyrrha

@Pyrrha I just discovered that the hard way, and it sucks that it silently fails like that.. – Saucier

Same here @KarthikT, the documentation was misleading ("may not exceed 30 days" indicates that 30 days should be allowed). – Pyrrha

Answer 6

Looks like some answers are not valid anymore.

I found out a key does not get set at all when the TTL is too high. For example 2992553564.

Tested with the following PHP code:

var_dump($memcached->set($id, "hello", 2992553564);  // true
var_dump($memcached->get($id));                      // empty!

var_dump($memcached->set($id, "hello", 500);  // true
var_dump($memcached->get($id));               // "hello"

Version is memcached 1.4.14-0ubuntu9.

Files

Zend framework limits lifetime to: 2592000. For the memcached backend, there is a lifetime limit of 30 days (2592000 seconds) – Annice

I wouldn't choose it now either but 10 years ago it was difference. framework.zend.com/issues/browse/ZF-4614 – Annice

@JohnMagnolia I'm sorry I might have gotten your comment wrong, thanks for the reference. I am not sure in 2019 why the keys I placed in 2017 did not get set with a timestamp from 2064 as written in the docs, it should accept any unix timestamp. – Files

"Expiration times are specified in unsigned integer seconds. They can be set from 0, meaning "never expire", to 30 days (60*60*24*30). Any time higher than 30 days is interpreted as a unix timestamp date." – Files

But most probably the following paragraph is related to this problem from 2017: For binary protocol an expiration must be unsigned. If a negative expiration is given to the ASCII protocol, it is treated it as "expire immediately". – Files

Answer 7

On laravel config.session.lifetime setting that if set to be an equivalent of 30days above, will be considered as a timestamp (this will give an error of token mismatch everytime assuming that memcached is used).

To answer, memcached expiration could be set anytime. (Laravel's default setting (on v5.0) will set you to an already expire timestamp). If you did not set it, the defualt will be used.

Lowpitched

Answer 8

If I don't provide an expiration time and the cache gets full, what happens?

If the expiration is not provided (or TTL is set to 0) and the cache gets full then your item may or may not get evicted based on the LRU algorithm.

Memcached provides no guarantee that any item will persist forever. It may be deleted when the overall cache gets full and space has to be allocated for newer items. Also in case of a hard reboot all the items will be lost.

From user internals doc

Items are evicted if they have not expired (an expiration time of 0 or some time in the future), the slab class is completely out of free chunks, and there are no free pages to assign to a slab class.

Below is how you can reduce the chance's of your item getting cleaned by the LRU job.

Create an item that you want to expire in a week? Don't always fetch the item but want it to remain near the top of the LRU for some reason? add will actually bump a value to the front of memcached's LRU if it already exists. If the add call succeeds, it means it's time to recache the value anyway.

source on "touch"

It is also good to monitor overall memory usage of memcached for resource planning and track the eviction statistics counter to know how often cache's are getting evicted due to lack of memory.

Baltic