How can an application store secrets in Google Cloud Datastore securely?

Question

I am building an application that will run on Google App Engine (GAE). It will need access to data stored by the user in other systems (e.g. the user's Nest thermostat, Yahoo mail). The application running on GAE will allow the user to provide credentials for the other system. The application will store these credentials in Google Cloud (Datastore) for later use by an application running on Google Compute Engine on the users behalf. The application will also allow OAuth to allow the user to allow the application access the external system in the user's behalf. The application will need to store user credentials (username and passwords) or OAuth access tokens in the Google Cloud.

The application will need to encrypt the secrets before they are stored and be able to unencrypt the data to send it to the external systems. That is, the system will need to use symmetric encryption and therefor need to securely manage keys.

How can the application store these secrets in the Google Cloud Datastore (Datastore) securely? I think I am looking for something like the AWS CloudHSM for Google. That is, I would like to store each secret with a seed and key id and use the key id to get the key from a key management system. This implementation would also allow for key rotation and other standard security practices.

I think I am looking for a Google Cloud service or Google API that provides secrets management and only allows an app with the proper Google app identifier to access the secrets.

Is there a service within Google Cloud or Google APIs that will manage secrets? Is there another architecture that I should be considering?

By the way, the application uses Google Identity Toolkit (GitKit) to authenticate and authorize users to use the GAE hosted application. The application allows users to create accounts using either federate identities or username and passwords via GitKit.

Thanks, chris

Aeolipile

Possible duplicate of GAE: best practices for storing secret keys? – Madi

Answer 1

In the meantime, Google also added a Key Management Service: https://cloud.google.com/kms/

You could e.g. use it to encrypt your data before storing it in a database. Or, use KMS to encrypt an AES key to encrypt your data, and possibly keep a backup of your AES key somewhere in case you lose access to KMS.

Exhibit

Answer 2

App Identity Service might be what you are looking for https://cloud.google.com/appengine/docs/java/appidentity/#Java_Asserting_identity_to_other_systems

It lets you sign content with an application-specific private key, and provides rotating certificates to validate signed content.

Thirty

Answer 3

So as far as I can tell the answer is that you can't. What you are looking for is an equivalent to KMS. That service let's you create and manage keys and do a bunch of your own crypto stuff. It's really cool and it will allow you to quickly do incredibly strong crypto with just a few simple lines of code. Azure has a similar service called KeyVault. It lacks automated key generation and rotation as far as I can tell, but other than that it's good. At the time of this response there was not an equivalent service for Google. They have an internal KMS which they used for crypto operations and you can provide your own keys, but that's pretty much it. Not quite the same thing that you get on KeyVault, and nothing like KMS.

That said there is hope. You can do one of two things:

1. Create a VPC and use an HSM from somewhere else. You could use RackSpace, or you could simply use AWS KMS. That sounds crazy but it's actually a good idea and the extra management is worth it. In general the most secure solution separate the keys from the encrypted data, particularly at rest. That means that keys in one data center and encrypted data stored in another data center is the most secure solution. That sounds like hard stuff, but thankfully I've made an opensource project which makes it very easy for you called KeyStor. With KeyStor you can get a data center that deals with encryption services set up in a day, no problem, and you can use AWS very cost effectively.
2. Set up your own cypto service, skip the HSM integration and simply be careful about who has access to the machines that maintain your keys. You can do this with KeyStor as well, and if KeyStor doesn't quite do what you want, that's why it's open-source. Take the code and build what you need to build.

Premonitory

Answer 4

You could store secrets in storage (e.g., in Datastore, Google Cloud Storage, or another storage system of your choice) and encrypt those with a key from Google's Cloud KMS.

Here's some documentation from Google on secret management, and here's a codelab on specifically encrypting data in Google Cloud Storage at the application layer using Cloud KMS.

Stag

Answer 5

For the Google Cloud managed service that provides the API for secure storage of secrets, see Google Cloud Secret Manager for more details.

Secret Manager is a secure and convenient storage system for API keys, passwords, certificates, and other sensitive data. Secret Manager provides a central place and single source of truth to manage, access, and audit secrets across Google Cloud.

Lineolate