How to delete keys matching a pattern in Redis Cluster

Question

I have tried method in this question, but it does not work since I'm working in cluster mode, and redis told me:

(error) CROSSSLOT Keys in request don't hash to the same slot

Exothermic

Some folks have suggested using Eval as an option; but i haven't been able to find a good example using Eval. Any leads? An example for some golang redis-client library would be even better. – Wareroom

Answer 1

Answers for that question try to remove multiple keys in a single DEL. However, keys matching the given pattern might NOT locate in the same slot, and Redis Cluster DOES NOT support multiple-key command if these keys don't belong to the same slot. That's why you get the error message.

In order to fix this problem, you need to DEL these keys one-by-one:

redis-cli --scan --pattern "foo*" |xargs -L 1 redis-cli del

The -L option for xargs command specifies the number of keys to delete. You need to specify this option as 1.

In order to remove all keys matching the pattern, you also need to run the above command for every master nodes in your cluster.

NOTE

1. With this command, you have to delete these keys one-by-one, and that might be very slow. You need to consider re-designing your database, and use hash-tags to make keys matching the pattern belong to the same slot. So that you can remove these keys in a single DEL.

2. Either SCAN or KEYS command are inefficient, especially, KEYS should not be used in production. You need to consider building an index for these keys.

Amplexicaul

I am getting (error) CROSSSLOT Keys in request don't hash to the same slot I am using following to delete pattern hi* form cluster redis-cli -u redis://localhost:6379 --scan --pattern 'hi*' | xargs redis-cli -u redis://localhost:6379 DEL – Ecclesiastic

@Ecclesiastic As I mentioned in the answer, you need to specify -L 1 parameter for the xargs command – Amplexicaul

Answer 2

Building on for_stack's answer, you can speed up mass deletion quite a bit using redis-cli --pipe, and reduce the performance impact with UNLINK instead of DEL if you're using redis 4 or higher.

redis-cli --scan --pattern "foo*" | xargs -L 1 echo UNLINK | redis-cli --pipe

Output will look something like this:

All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 107003

You do still need to run this against every master node in your cluster. If you have a large number of nodes, it's probably possible to automate the process further by parsing the output of CLUSTER NODES.

Interplay

Answer 3

redis-cli provides a -c option to follow MOVED redirects. However, it should be deleted one at a time because you cannot guarantee two keys will be in the same node.

redis-cli -h myredis.internal --scan --pattern 'mycachekey::*' | \
  xargs -L 1 -d'\n' redis-cli -h myredis.internal -c del

The first part provides a list of keys --scan prevents Redis from locking. xargs -L 1 runs the command for one entry at a time. -d'\n' disables the processing of quotes so you can have quoted strings like "SimpleKey[hello world]" be passed to the command otherwise the spaces will make it have two keys.

Waldrop