Connect to Memorystore from Cloud Run

Question

I want to run a service on Google Cloud Run that uses Cloud Memorystore as cache.

I created an Memorystore instance in the same region as Cloud Run and used the example code to connect: https://github.com/GoogleCloudPlatform/golang-samples/blob/master/memorystore/redis/main.go this didn't work.

Next I created a Serverless VPC access Connectore which didn't help. I use Cloud Run without a GKE Cluster so I can't change any configuration.

Is there a way to connect from Cloud Run to Memorystore?

Galang

Answer 1

To connect Cloud Run (fully managed) to Memorystore you need to use the mechanism called "Serverless VPC Access" or a "VPC Connector".

As of May 2020, Cloud Run (fully managed) has Beta support for the Serverless VPC Access. See Connecting to a VPC Network for more information.

Alternatives to using this Beta include:

• Use Cloud Run for Anthos, where GKE provides the capability to connect to Memorystore if the cluster is configured for it.
• Stay within fully managed Serverless but use a GA version of the Serverless VPC Access feature by using App Engine with Memorystore.

Orang

Are there any updates on this VPC Connector support feature? (Oct 2019) – Perennial

Not yet, I will update when something is announced. – Orang

So there is no way to use Memorystore on Cloud Run? :/ – Wager

@Orang Any rough ETA when we can expect this support to be available? – Sensualist

This just reached beta out ahead of the documentation. Once the docs are out I'll update the answer. – Orang

Works like a charm (although you need to pay for Serverless VPC Access). – Humoresque

Answer 2

While waiting for serverless VPC connectors on Cloud Run - Google said yesterday that announcements would be made in the near term - you can connect to Memorystore from Cloud Run using an SSH tunnel via GCE.

The basic approach is the following.

First, create a forwarder instance on GCE

gcloud compute instances create vpc-forwarder --machine-type=f1-micro --zone=us-central1-a

Don't forget to open port 22 in your firewall policies (it's open by default).

Then install the gcloud CLI via your Dockerfile

Here is an example for a Rails app. The Dockerfile makes use of a script for the entrypoint.

# Use the official lightweight Ruby image.
# https://hub.docker.com/_/ruby
FROM ruby:2.5.5

# Install gcloud
RUN curl https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz > /tmp/google-cloud-sdk.tar.gz
RUN mkdir -p /usr/local/gcloud \
  && tar -C /usr/local/gcloud -xvf /tmp/google-cloud-sdk.tar.gz \
  && /usr/local/gcloud/google-cloud-sdk/install.sh
ENV PATH $PATH:/usr/local/gcloud/google-cloud-sdk/bin

# Generate SSH key to be used by the SSH tunnel (see entrypoint.sh)
RUN mkdir -p /home/.ssh && ssh-keygen -b 2048 -t rsa -f /home/.ssh/google_compute_engine -q -N ""

# Install bundler
RUN gem update --system
RUN gem install bundler

# Install production dependencies.
WORKDIR /usr/src/app
COPY Gemfile Gemfile.lock ./
ENV BUNDLE_FROZEN=true
RUN bundle install

# Copy local code to the container image.
COPY . ./

# Run the web service on container startup.
CMD ["bash", "entrypoint.sh"]

Finally open an SSH tunnel to Redis in your entrypoint.sh script

# !/bin/bash

# Memorystore config
MEMORYSTORE_IP=10.0.0.5
MEMORYSTORE_REMOTE_PORT=6379
MEMORYSTORE_LOCAL_PORT=6379

# Forwarder config
FORWARDER_ID=vpc-forwarder
FORWARDER_ZONE=us-central1-a

# Start tunnel to Redis Memorystore in background
gcloud compute ssh \
  --zone=${FORWARDER_ZONE} \
  --ssh-flag="-N -L ${MEMORYSTORE_LOCAL_PORT}:${MEMORYSTORE_IP}:${MEMORYSTORE_REMOTE_PORT}" \
  ${FORWARDER_ID} &

# Run migrations and start Puma
bundle exec rake db:migrate && bundle exec puma -p 8080

With the solution above Memorystore will be available to your application on localhost:6379.

There are a few caveats though

1. This approach requires the service account configured on your Cloud Run service to have the roles/compute.instanceAdmin role, which is quite powerful.
2. The SSH keys are backed into the image to speedup container boot time. That's not ideal.
3. There is no failover if your forwarder crashes.

I've written a longer and more elaborated approach in a blog post that improves the overall security and adds failover capabilities. The solution uses plain SSH instead of the gcloud CLI.

Cowbird

Answer 3

If you need something in your VPC, you can also spin up Redis on Compute Engine

It's more costly (especially for a Cluster) than Redis Cloud - but an temp solution if you have to keep the data in your VPC.

Convery

Links to external resources are encouraged, but please add context around the link so your fellow users will have some idea what it is and why it’s there. Always quote the most relevant part of an important link, in case the target site is unreachable or goes permanently offline. See: How to anwser. – Jellied

Since Google Cloud Run containers cannot (yet) connect to a VPC, I don't see how this would help in this case. – Sensualist