Get all keys in Redis cluster

Question

I am using Redis cluster version redis-5.0.5. I want to see all the keys present in my Redis cluster. I know for standalone we use KEYS * to get all the keys.

what is the way to see all keys in Redis cluster?

$ redis-cli -h hostname -p 90001 -c 
hostname:90001> KEYS *
(empty list or set)

// I have data on my cluster 

Fable

Answer 1

Basically, you'd need to run KEYS * (not in production, please!) on every one of the nodes. The cli can do this with the '--cluster call' command, like so:

redis-cli --cluster call hostname:90001 KEYS "*"

Susurration

I have 10 hosts. are there some commands coming in next release? – Fable

You need to call this only once - the --cluster call executes the specified command on each of the nodes in the cluster. – Susurration

call argus seems great. but not works on windows redis-cli.exe 3.2.100. – Chairmanship

@ItamarHaber this works fine for me. however, I noticed, this is returning 2*no_of_keys as this query is including both master and slave. – Fable

@Fable thanks for bumping this, I was actually aware of this shortcoming and you reminded me about the relevant pull request that addresses it (github.com/redis/redis/pull/6491) so you can expect it to be available in the next versions. – Susurration

How to use --cluster call option in RedisInsight-v2? – Talaria

redis-cli --cluster help said you can use --cluster-only-masters for call which likely mean only target master node. – Hammy

Answer 2

Requirement:

1. redis-cli
2. awk
3. grep
4. xargs

May be can try this assuming redis server reside in localhost with the default port 6379:

redis-cli cluster nodes | awk '{print $2" "$3}' | grep master | awk -F @ '{print $1}' | awk -F : '{print " -h "$1" -p "$2" --scan"}' | xargs -L 1 redis-cli -c

Longer version base question above (90001 port number seriously?) and also you can change the pattern (* no filter) for filtering certain key pattern:

redis-cli -h hostname -p 90001 cluster nodes | awk '{print $2" "$3}' | grep master | awk -F @ '{print $1}' | awk -F : '{print " -h "$1" -p "$2" --scan --pattern *"}' | xargs -L 1 redis-cli -c

It connects to any one of the redis node to get cluster info and then execute the keys scanning command on each of the master node.

Hammy

The complex pipeline is rather ghastly; perhaps see useless grep though refactoring it properly will probably require access to the output from redis-cli cluster nodes so we can see what parts exactly you are picking out from there. – Cytokinesis

Answer 3

The SCAN command may be what you're looking for, but it's O(N) so the more keys you have, the slower it's going to be. Also, check out this answer by Marc Gravell for another approach using sets: Get values by key pattern in StackExchange.Redis

Gridley

Is there a database where adding more data is not going to be at least O(N) on a query like this? – Cahoon

SCAN also suffers the same issue of the KEYS. It will only return the keys from the scanned node hit by your load balancer so not an option – Gaylegayleen

Answer 4

You can use the following command to scan for all keys across nodes in your Redis cluster:

redis-cli -h localhost CLUSTER NODES \
  | grep master \
  | awk '{print $2}' \
  | cut -f1 -d '@' \
  | xargs -I '{}' redis-cli -u redis://{} --scan --pattern '*'

TL;DR: Get all the IP and port of all nodes in the cluster and scan for keys on each of them.

1. redis-cli -h localhost CLUSTER NODES obtains all information from the cluster configuration, including the node addresses to connect to for running commands
2. grep master selects only the master nodes
3. awk '{print $2}' selects the second column containg IP, port and cluster bus port
4. cut -f1 -d '@' to truncate the cluster bus port
5. xargs in combination with redis-cli to run the desired command. The -u flag makes it possible to provide server address and port in one string

Similarily you can also execute node-specific commands on every cluster nodes, see here.

---

Also, in the above command, make sure to replace localhost with your actual host address, provide a port with -p if necessary. Keep in mind, while the cost of scanning keys is less than KEYS *, it can still impact performance.

Cung

Answer 5

Some times you want to get the keys those match a pattern and output as an unique list and can be passed to a command like DEL to delete them. You may use the following command:

redis-cli --cluster call REDIS_CLUSTER_ENDPOINT keys "pattern*"| grep -v "Calling keys" | cut -d ":" -f3 | grep "\S" | sort -u

How it works

redis-cli --cluster call REDIS_CLUSTER_ENDPOINT keys "pattern*": the command to scan keys with pattern "pattern*" in all nodes.
      grep -v "Calling keys"                                   : remove debug line that starts with Calling keys.
                cut -d ":" -f3                                 : print the third column which are the keys
                      grep "\S"                                : remove empty line
                            sort -u                            : output an unique list (since a same key can be found in multiple nodes)

DISCLAIMER: I did not test it in production with millions keys so please be aware when using it.

Cordell