How to delete items with same prefix key in memcached?

Question

For example, I have some cached items with same prefix, such as

'app_111111', 'app_222222', 'app_333333', ...

Can I remove such 'app_xxxxxx' items by any memcached commands?

Keyway

https://mcmap.net/q/442335/-memcache-and-wildcards – Kura

Answer 1

Memcached does not offer this functionality out of the box so you have to build it in yourself.

The way I solve this is by defining a prefix (or namespace) in my application for groups of keys. Any key that I set in memcached has that prefix before it. Whenever I want to "delete" stuff from Memcached, I just change the prefix. And whenever I want to lookup a key in Memcached, I add that prefix to it.

In your case, you could start by setting the prefix to, say, MyAppPrefix1, so your keys will be stored as MyAppPrefix1::app_333333, MyAppPrefix1::app_444444.

Later on when you want to "delete" these entries, set your application to use MyAppPrefix2. Then, when you try to get a key from Memcached called app_333333, it will look for MyAppPrefix2::app_333333 and will not find it the first time around, as if it had been deleted.

Fulgor

And how do you delete old entries? by time expiration? – Dripstone

@AlfonsoFernandez-Ocampo yes, why not? Memcache will delete them when needed – Eikon

that looks excellent, but, Memcached will delete after the expiration time or when it is about to be full? – Bouffard

Answer 2

How about this function in php:

function deletekeysbyindex($prefix) {
    $m = new Memcached();
    $m->addServer('localhost', 11211);
    $keys = $m->getAllKeys();
    foreach ($keys as $index => $key) {
        if (strpos($key,$prefix) !== 0) {
            unset($keys[$index]);
        } else {
            $m->delete($key);
        }
    }
    return $keys;
}

Deletes keys beginning with $prefix and returns a list of all keys removed. I ran this on 30,000+ keys just now on a shared server and it was pretty quick - probably less than one second.

Thigmotropism

This is not guaranteed to work. php.net/manual/en/memcached.getallkeys.php "As memcache doesn't guarantee to return all keys you also cannot assume that all keys have been returned. " – Gorey

Thanks for pointing that out @TimMartens. I suppose it's about as good as you can get? I wish there were a bit more info about when and why the getAllKeys() method is likely to come up short.. can you shed any light on this? The docs are pretty unhelpful. – Thigmotropism

Answer 3

This is a hack that works, albeit a bit slow. On a server with 0.6 million keys, it took half a second to complete.

    $prefix = 'MyApp::Test';
    $len = strlen($prefix);

    $proc = popen('/usr/local/bin/memdump --servers=localhost', 'r');
    while (($key = fgets($proc)) !== false) {
        if (substr_compare($key, $prefix, 0, $len) === 0) {
            $memcached->delete(substr($key, 0, -1));
        }
    }

Opportunity

Answer 4

We can not do that in only one request to memcache. We just can do this:

public function clearByPrefix($prefixes = array()) {
    $prefixes = array_unique($prefixes);

    $slabs = $this->memcache->getExtendedStats('slabs');
    foreach ($slabs as $serverSlabs) {
        if ($serverSlabs) {
            foreach ($serverSlabs as $slabId => $slabMeta) {
                if (is_int($slabId)) {
                    try {
                        $cacheDump = $this->memcache->getExtendedStats('cachedump', (int) $slabId, 1000);
                    } catch (Exception $e) {
                        continue;
                    }

                    if (is_array($cacheDump)) {
                        foreach ($cacheDump as $dump) {
                            if (is_array($dump)) {
                                foreach ($dump as $key => $value) {

                                    $clearFlag = false;
                                    // Check key has prefix or not
                                    foreach ($prefixes as $prefix) {
                                        $clearFlag = $clearFlag || preg_match('/^' . preg_quote($prefix, '/') . '/', $key);
                                    }
                                    // Clear cache
                                    if ($clearFlag) {
                                        $this->clear($key);
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

And call this function like this:

        $prefixes = array();

        array_push($prefixes, 'prefix1_');
        array_push($prefixes, 'prefix2_');
        array_push($prefixes, 'prefix3_');

        $this->clearByPrefix($prefixes);

Spoonerism

Even if your code seems easy to understand, you could explain what it does and why you believe this solution helps the asker. By the way, preg_match when you could just call strpos, is really an overkill. – Featured

Maybe I should add more comments in my code :) About preg_match, I don't think that it's an overkill. You know, it's just a simple thing for this case. I even intend that the regex will be more complex in the future. :) Thanks for your comment bro! – Spoonerism