Accessing GCP Memorystore from local machines

Question

Whats the best way to access Memorystore from Local Machines during development? Is there something like Cloud SQL Proxy that I can use to set up a tunnel?

Tameika

it's disappointing that you can't connect to this cloud redis instance without paying to create another intermediate instance – Careerism

Answer 1

You can spin up a Compute Engine instance and use port forwarding to connect to your Redis machine.

For example if your Redis machine has internal IP address 10.0.0.3 you'd do:

gcloud compute instances create redis-forwarder --machine-type=f1-micro
gcloud compute ssh redis-forwarder -- -N -L 6379:10.0.0.3:6379

As long as you keep the ssh tunnel open you can connect to localhost:6379

Update: this is now officially documented: https://cloud.google.com/memorystore/docs/redis/connecting-redis-instance#connecting_from_a_local_machine_with_port_forwarding

Bluster

I can create instance but while running the second command I am getting this below error : ERROR: (gcloud.compute.ssh) Could not SSH into the instance. It is possible that your SSH key has not propagated to the instance yet. Try running this command again. If you still cannot connect, verify that the firewall and instance are set to accept ssh traffic. – Brainwork

This is great . – Analysand

Why isn't this the accepted answer? en.wikipedia.org/wiki/Jump_server – Dur

Just used this yesterday. Was literally as easy as installing the GCP tool and running these two commands. Thanks. – Tingly

Hmm, when I run the second command the CLI just hangs forever. Any ideas what it might be? – Excurrent

@Excurrent Might need to check if the GCE instance is running in the same zone as your Redis instance. – Licketysplit

redis-cli does connect, but if I try to run any command like get <key> or cluster info I get Error: Connection reset by peer – Bernardobernarr

Beautiful! Wish Google had this documented. – Iny

@Bluster should we take into consideration also the machine type for bandwidth ingress and egress right? – Flocculate

@Flocculate yes, if you need a lot of bandwidth for this tunnel you might want a bigger machine than an f1-micro. – Bluster

Doesn't this approach increase the time required to fetch the data from Redis instead of a direct connection to the REDIS server? – Fetus

@Shriram probably. But a direct connection isn't possible since Redis isn't exposed on an external IP. Also for dev use cases the extra 1ms or so probably doesn't matter. – Bluster

@Excurrent you have to make sure you create the redis-forwarder in the same network as your redis-network. Check in the memorystore console and copy the Authorized Network name. then add it with gcloud compute instances create redis-forwarder --network-interface network=[Authorized Network] --machine-type=f1-micro – Grant

Answer 2

I created a vm on google cloud

gcloud compute instances create redis-forwarder --machine-type=f1-micro

then ssh into it and installed haproxy

sudo su
apt-get install haproxy

then updated the config file

/etc/haproxy/haproxy.cfg

....existing file contents
frontend redis_frontend
  bind *:6379
  mode tcp
  option tcplog
  timeout client  1m
  default_backend redis_backend

 backend redis_backend
   mode tcp
   option tcplog
   option log-health-checks
   option redispatch
   log global
   balance roundrobin
   timeout connect 10s
   timeout server 1m
   server redis_server [MEMORYSTORE IP]:6379 check

restart haproxy

/etc/init.d/haproxy restart

I was then able to connect to memory store from my local machine for development

Severson

Answer 3

You can spin up a Compute Engine instance and setup an haproxy using the following docker image haproxy docker image then haproxy will forward your tcp requests to memorystore.

For example i want to access memorystore instance with ip 10.0.0.12 so added the following haproxy configs:

frontend redis_frontend
   bind *:6379
   mode tcp
   option tcplog
   timeout client  1m
   default_backend redis_backend

backend redis_backend
   mode tcp
   option tcplog
   option log-health-checks
   option redispatch
   log global
   balance roundrobin
   timeout connect 10s
   timeout server 1m
   server redis_server 10.0.0.12:6379 check

So now you can access memorystore from your local machine using the following command:

redis-cli -h <your-haproxy-public-ipaddress> -p 6379

Note: replace with you actual haproxy ip address.

Hope that can help you to solve your problem.

Dentalium

Answer 4

This post builds on earlier ones and should help you bypass firewall issues.

Create a virtual machine in the same region(and zone to be safe) as your Memorystore instance. On this machine:

• Add a network tag with which we will create a firewall rule to allow traffic on port 6379
• Add an external IP with which you will access this VM

SSH into this machine and install haproxy

sudo su
apt-get install haproxy

add the following below existing config in the /etc/haproxy/haproxy.cfg file

frontend redis_frontend
   bind *:6379
   mode tcp
   option tcplog
   timeout client  1m
   default_backend redis_backend

backend redis_backend
   mode tcp
   option tcplog
   option log-health-checks
   option redispatch
   log global
   balance roundrobin
   timeout connect 10s
   timeout server 1m
   server redis_server [MEMORYSTORE IP]:6379 check

restart haproxy

/etc/init.d/haproxy restart

Now create a firewall rule that allows traffic on port 6379 on the VM. Ensure:

• It has the same target tag as the networking tag we created on the VM.
• It allows traffic on port 6379 for the TCP protocol.

Now you should be able to connect remotely like so:

redis-cli -h [VM IP] -p 6379

Nihhi

Answer 5

Like @Christiaan answered above, it almost worked for me but I needed a few other things to check to make it work well.

• Firstly, in my case, my Redis is running in a specific network other than default network, so I had to create the jumpbox inside the same network (let's call it my-network)
• Secondly, I needed to apply a firewall rule to open port 22 in that network.

So putting all my needed command it looks like this:

gcloud compute firewall-rules create default-allow-ssh --project=my-project  --network my-network --allow tcp:22 --source-ranges 0.0.0.0/0

gcloud compute instances create jump-box --machine-type=f1-micro --project my-project --zone europe-west1-b --network my-network

gcloud compute ssh jump-box  --project my-project --zone europe-west1-b -- -N -L 6379:10.177.174.179:6379

Then I have access to Redis locally on 6379

Yoheaveho

Answer 6

Memorystore does not allow connecting from local machines, other ways like from CE, GAE are expensive especially your project is small or in developing phase, I suggest you create a cloud function to execute memorystore, it's serverless service which means lower fee to execute. I wrote small tool for this, the result is similar to run on local machine. You can check if help to you.

Tread