Multiple K8S containers connecting to Google Cloud SQL through proxy

Question

I would like to connect my Kubernetes cluster to Google Cloud SQL.

I have at least 10 different deployed pods which presently connect to MySQL [docker image deployed to k8s] using a JDBC url + username/password.

It it possible to use a single instance of the Google Cloud SQL Proxy and connect all the pods through this proxy to the Cloud SQL database? Ideally I would like to replace the mysql running in the container with the proxy.

I would prefer not having to run the proxy inside each deployment. The only samples I found seem to indicate the proxy needs to be declared in each deployment.

Amparoampelopsis

Answer 1

I found a solution.

Deploy the proxy with the yml below, and expose the deployment as a service. Most importantly, make the proxy listen on 0.0.0.0, instead of default 127.0.0.1. All the secrets as per the Google Cloud sql documentation

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: mysql
spec:
  replicas: 1
  template:
    metadata:
      name: mysql
      labels:
        name: mysql
    spec:
      containers:
         - image: b.gcr.io/cloudsql-docker/gce-proxy:1.05
           name: cloudsql-proxy
           command: ["/cloud_sql_proxy", "--dir=/cloudsql",
                     "-instances=MYSQL:ZONE:DATABASE_INSTANCE=tcp:0.0.0.0:3306",
                     "-credential_file=/secrets/cloudsql/credentials.json"]
           volumeMounts:
             - name: cloudsql-oauth-credentials
               mountPath: /secrets/cloudsql
               readOnly: true
             - name: ssl-certs
               mountPath: /etc/ssl/certs
           ports:
             - containerPort: 3306
               name: mysql
      volumes:
        - name: cloudsql-oauth-credentials
          secret:
            secretName: cloudsql-oauth-credentials
        - name: ssl-certs
          hostPath:
            path: /etc/ssl/certs

The solution is slightly more expensive than having the proxy in the same deployment as the client software, since there is an extra TCP connection.

However there are many benefits:

• Much simpler and doesn't require modifying existing K8S deployment files
• Allows switching the implementation to a MySQL Docker container or using the Google Cloud SQL proxy without any modifications to the client configuration.

Amparoampelopsis

I'm intrigued @Hylton Peimer - I'm faced with the same problem: I need to host the SQL Proxy in a separate pod, but how are you making your two pods communicate with each other? I either get connection refused or host name not found – Edithe

Expose a Kubernetes Service for the deployment. Then use the name of the service. For example above "kubectl expose deployment mysql" and then have the clients use "mysql" as you would the hostname. – Amparoampelopsis

While I share the same problem with you, there are some good reasons for running the proxy as a sidecar container in the same pod: - By exposing the proxy to the cluster, you lose control over which services have access to it, unless you protect it some other way - Without using labels and selectors, you can't be sure that there is a proxy pod running on each node that needs it, which could affect performance depending on which client needs to talk to the server and if that client happens to be on the same node as with the proxy. – Konikow

Changing the default tcp:5432 to tcp:0.0.0.0:5432 did it for me, since tcp:5432 apparently translates to tcp:127.0.0.1:5432 implicitly – Uralite

Consider using the Private IP of Google Cloud SQL. This way you no longer need the proxy, directly connect to the SQL instance from inside K8S. – Amparoampelopsis

Answer 2

You can create a deployment and a service to expose the cloudsql proxy to other pods like so:

apiVersion: v1
kind: Service
metadata:
  name: cloudsqlproxy
spec:
  ports:
  - port: 3306
    targetPort: database-port
  selector:
    app: cloudsqlproxy
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: cloudsqlproxy
spec:
  template:
    metadata:
      labels:
        app: cloudsqlproxy
    spec:
      volumes:
      - name: service-account-token
        secret:
          secretName: service-account-token
      containers:
      - name: cloudsql-proxy
        image: gcr.io/cloudsql-docker/gce-proxy:1.11
        imagePullPolicy: Always
        command:
        - /cloud_sql_proxy
        - -instances=<project>:<cloudsqlinstance>=tcp:0.0.0.0:3306
        - -credential_file=/secrets/cloudsql/credentials.json
        ports:
        - name: database-port
          containerPort: 3306
        volumeMounts:
        - name: service-account-token
          mountPath: /secrets/cloudsql
          readOnly: true

So within any of your pods, the database your MYSQL_HOST:MYSQL_PORT will be cloudsqlproxy:3306

For multiple databases through the same proxy, you'd have the same deployment structure for the proxy, except that you will now expose 2 ports from the pod, like so:

apiVersion: extensions/v1beta1
...
spec:
  template:
    ...
    spec:
      volumes:
      ...
      containers:
      - name: cloudsql-proxy
        ...
        ports:
        - name: database-port1
          containerPort: 3306
        - name: database-port2
          containerPort: 3307
        ...

Then you'd create 2 services to for discovery on those ports like so:

apiVersion: v1
kind: Service
metadata:
  name: cloudsqlproxy-db1
spec:
  ports:
  - port: 3306
    targetPort: database-port1
  selector:
    app: cloudsqlproxy
---
apiVersion: v1
kind: Service
metadata:
  name: cloudsqlproxy-db2
spec:
  ports:
  - port: 3306
    targetPort: database-port2
  selector:
    app: cloudsqlproxy

So, with both services set to port 3306, you can connect to each database on that port:

mysql --host=cloudsqlproxy-db1 --port=3306 ...
mysql --host=cloudsqlproxy-db2 --port=3306 ...

Reference: https://github.com/GoogleCloudPlatform/cloudsql-proxy/blob/master/Kubernetes.md

Iconoscope

This is awesome & should be marked as the accepted answer – Mistreat

Answer 3

With Google "Private IP" the cloud proxy is now irrelevant!

Amparoampelopsis