Terraform RDS database credentials

Question

I am trying to use AWS secrets manager to declare RDS admin credentials.

1. Declared credentials in rds.tf in variable RdsAdminCred as key/value pair
2. Declared secret as well in the same tf file

variable "RdsAminCred" {
    default = {
        username = "dbadmin"
        password = "dbadmin#02avia"
    }
    type = map(string)
}

resource "aws_secretsmanager_secret" "RdsAminCred" {
  name = "RdsAminCred"
}
resource "aws_secretsmanager_secret_version" "RdsAminCred" {
  secret_id     = aws_secretsmanager_secret.RdsAminCred.id
  secret_string = jsonencode(var.RdsAminCred)
}

3. I am not sure how to use the secret string in the declaration below, to replace the hardcoded value for username and password.

resource "aws_db_instance" "default" {
  identifier            = "testdb"
  allocated_storage    = 20
  storage_type         = "gp2"
  engine               = "mysql"
  engine_version       = "5.7"
  instance_class       = "db.t2.medium"
  name                 = "mydb"
 
  username             = "dbadmin"
  password             = "dbadmin#01avia"

Any help is appreciated..

Sooty

Can you clarify what do you want to achieve? If you hard-code your passwords in RdsAminCred, there is not much sense for secret_manager anyway, because the password will end up in your source code, worse, in public repo on github or gitlab. – Clamor

That was going to be next step , which I am not sure how to accomplish either. In the code above, I was trying to use Secrets Manager first. Ultimately the password should not be hardcoded in tf. So tI think he final approach would be use env variable (outside tf) or some other better approach and use AWS Secrets manager – Sooty

Answer 1

I would have a TF config that sets up your secret and stores it in AWS Secrets Manager, like this.

resource "random_password" "master"{
  length           = 16
  special          = true
  override_special = "_!%^"
}

resource "aws_secretsmanager_secret" "password" {
  name = "test-db-password"
}

resource "aws_secretsmanager_secret_version" "password" {
  secret_id = aws_secretsmanager_secret.password.id
  secret_string = random_password.master.result
}

And then in a separate TF config for your database, you can use the secret from AWS Secrets Manager.

data "aws_secretsmanager_secret" "password" {
  name = "test-db-password"

}

data "aws_secretsmanager_secret_version" "password" {
  secret_id = data.aws_secretsmanager_secret.password
}

resource "aws_db_instance" "default" {
  identifier            = "testdb"
  allocated_storage    = 20
  storage_type         = "gp2"
  engine               = "mysql"
  engine_version       = "5.7"
  instance_class       = "db.t2.medium"
  name                 = "mydb"
 
  username             = "dbadmin"
  password             = data.aws_secretsmanager_secret_version.password

In the comments above, Asri Badlah suggested that the password be entered manually in the console. And I guess you can do that. However, that approach does start to get away from the fundamental tenet of IaC - put everything in source control. Of course you don't want to check passwords, private keys or the like into source control. But here, you can see we're not doing that. We populate a secret with one config and consume it with another. This ensures the code can be checked into source control, but the password is not.

In terms of state, it's true that the password will be stored and decipherable in TF state. But if you use proper state management, this shouldn't be an issue. Ideally, you would want to be using remote state, encrypted and with restricted access.

As a final point, I would not use just random_password as Evan Closson suggested. That approach would mean that your database password is 100% managed by Terraform. By using Secrets Manager, your password is managed by a service, which means that you can do other stuff like rotate the password (not shown) and retrieve the password without having to rely on Terraform (e.g. terraform output or cracking open the state file).

Monaxial

When supplying the value to the database config, why do we need to specify it using a data resource? Does using aws_secretsmanager.secret_version.password.secret_string instead (i.e., using the value from the non data resource) not work correctly? – Gambell

@MattHancock as far as I understand, when you use the data block, you are telling terraform to fetch data for an existing resource in AWS instead of telling terraform to create that resource for you. – Watchword

One issue I'm having with this is that if I try to launch all at once, terraform tries to create the RDS instance first, which fails because the secret has not been created yet. Adding a 'dependsOn' field to the RDS pointing to both aws_secretsmanager_secret aws_secretsmanager_secret_version didn't help either. I had to comment out the RDS instance, appy to create the secret, add the rds instance back in. Not sure how this will play with a destroy operation. – Watchword

also noticed errors in the answer (edit queue is full, so can't edit myself) - 1) secret_id = data.aws_secretsmanager_secret.password.id (was missing .id) 2) password = data.aws_secretsmanager_secret_version.password won't work because data.aws_secretsmanager_secret_version.password is an object, not a value – Watchword

This is a great way to bootstrap the AWS secret manager secret in Terraform and then use it in the RDS resource. We wanted to only set the random generated value on initial creation and update the value outside of terraform. I found the lifecycle ignore_changes solution in https://mcmap.net/q/503344/-how-to-tell-terraform-to-skip-the-secret-manager-resource-if-it-exists very helpful in combination with this approach. – Tortfeasor

First, the secret_string is missing: it should be data.aws_secretsmanager_secret_version.password.secret_string. Second, the problem with this solution is that it does store the password in git, as the terraform.tfstate file will contain the password (and you do want to have terraform.tfstate in git). – Kerry

> and you do want to have terraform.tfstate in git No. Git is not a good place for state. Terraform has several remote state options, like S3. Use those. developer.hashicorp.com/terraform/language/state/remote – Westmorland

Answer 2

I'd recommend using the random_password resource instead. Then you can reference that in the cluster configuration and secrets manager.

Example:

resource "random_password" "master_password" {
  length  = 16
  special = false
}

resource "aws_rds_cluster" "default" {
  cluster_identifier = "my-cluster"
  
  master_username = "admin"
  master_password = random_password.default_master_password.result

  # other configurations
  # .
  # .
  # .
}

resource "aws_secretsmanager_secret" "rds_credentials" {
  name = "credentials"
}

resource "aws_secretsmanager_secret_version" "rds_credentials" {
  secret_id     = aws_secretsmanager_secret.rds_credentials.id
  secret_string = <<EOF
{
  "username": "${aws_rds_cluster.default.master_username}",
  "password": "${random_password.master_password.result}",
  "engine": "mysql",
  "host": "${aws_rds_cluster.default.endpoint}",
  "port": ${aws_rds_cluster.default.port},
  "dbClusterIdentifier": "${aws_rds_cluster.default.cluster_identifier}"
}
EOF
}

Kameko

This is the best answer, because that associative array in the secret_string is exactly the output generated by having SSM generate "RDS Credentials" anyway (although, it does store it as a "Key-Value" secret instead of a string in the console itself). – Greenfinch

Isn't an issue with this approach that any time you run an apply after the initial apply, it's going to destroy & re-create your RDS instance because the random password is going to change each time? – Washing

Answer 3

variable "RdsAdminCred" {
  default = {
    username = "dbadmin"
    password = "dbadmin#02avia"
  }
  type = map(string)
}

resource "aws_secretsmanager_secret" "RdsAdminCred" {
  name = "RdsAdminCred"
}
resource "aws_secretsmanager_secret_version" "RdsAdminCred" {
  secret_id     = aws_secretsmanager_secret.RdsAdminCred.id
  secret_string = jsonencode(var.RdsAdminCred)
}

after you have created a secret, you need to take data from there

data "aws_secretsmanager_secret" "env_secrets" {
  name = "RdsAdminCred"
  depends_on = [
    aws_secretsmanager_secret.RdsAdminCred
  ]
}
data "aws_secretsmanager_secret_version" "current_secrets" {
  secret_id = data.aws_secretsmanager_secret.env_secrets.id
}
resource "aws_db_instance" "default" {
  identifier        = "testdb"
  allocated_storage = 20
  storage_type      = "gp2"
  engine            = "mysql"
  engine_version    = "5.7"
  instance_class    = "db.t2.medium"
  name              = "mydb"

  username = jsondecode(data.aws_secretsmanager_secret_version.current_secrets.secret_string)["username"]
  password = jsondecode(data.aws_secretsmanager_secret_version.current_secrets.secret_string)["password"]
}

Herbage

I would recommend to set the sensitive = true parameter on your variable. See also the official documentation – Coolish

Additionally you could put the password into locals and also make the jsondecode sensitive and hide its output. sensitive(jsondecode(data.aws_secretsmanager_secret_version.current_secrets.secret_string)["password"]). See the sensitive function documentation. – Coolish

Answer 4

AWS terraform aws_db_instance resource has a new feature for doing that easily.

You can specify the manage_master_user_password attribute to enable managing the master password with Secrets Manager. You can also update an existing cluster to use Secrets Manager by specify the manage_master_user_password attribute and removing the password attribute (removal is required).

NOTE: By default the secret rotation is enabled which means the DB secrets will be updated weekly, please disable this if the behaviour is not expected.

It is possible to use default account KMS key by removing the master_user_secret_kms_key_id attribute.

resource "aws_kms_key" "example" {
  description = "Example KMS Key"
}

resource "aws_db_instance" "default" {
  allocated_storage             = 10
  db_name                       = "mydb"
  engine                        = "mysql"
  engine_version                = "5.7"
  instance_class                = "db.t3.micro"
  manage_master_user_password   = true
  master_user_secret_kms_key_id = aws_kms_key.example.key_id
  username                      = "foo"
  parameter_group_name          = "default.mysql5.7"
}

source: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance

Grenville

How can you retrieve the key once set? The state file does not appear to have it, or rather, the key ID there doesn't appear to correspond to any secret. – Pentachlorophenol

You can find the kms key in KMS page in the console and the DB secrets will be in the Secrets Manager page in AWS console. – Grenville

Answer 5

In your Terraform code, you can use the aws_secretsmanager_secret_version data source to read this secret:

data "aws_secretsmanager_secret_version" "creds" {
  # write your secret name here
  secret_id = "your_secret"
}

parse the secret from JSON, using jsondecode :

locals {
  your_secret = jsondecode(
    data.aws_secretsmanager_secret_version.creds.secret_string
  )
}

Now pass the secret to RDS:

resource "aws_db_instance" "example" {
  engine               = "engine"
  engine_version       = "version"
  instance_class       = "instance"
  name                 = "example"
  # Set the secrets from AWS Secrets Manager
  username = local.your_secret.username
  password = local.your_secret.password
}

Impendent

I tried to include the code above (that declares locals ) and rerun, but I got an error A data source "aws_secretsmanager_secret_manager" "RdsAdminCred" has not been declared in the root module – Sooty

I think it is not a good idea to create your secret using terraform, since it contains a sencitve data it should be not included e in your code, alternatively create the secret using secret manager first, then reference this secret from terraform. – Impendent

You are suggesting to use the Secrets Manager in AWS Console to create the secret , correct ? – Sooty

Yes correct, then read this secret using terraform – Impendent

Advantages of this technique Keep plain text secrets out of your code and version control system. Your secrets are stored in an encrypted format in version control – Impendent

How can we automate the step of secret creation ? How to apply infrastructure as code principle to the secret creation ? – Sooty

you may create the secret with terraform and set a value manually. Otherwise your secret will be visible in the terraform state file and in your source control, – Impendent

@AsriBadlah, your secrets will be stored in plain text in terraform state file, even if you reference a manually created secret. It is kind of annoying but I don't see any way to set this password and not have its value on the state file. – Thanks

Answer 6

For those using terraform-aws-modules/rds-aurora/aws module, there is an option manage_master_user_password_rotation which if set to true, credentials are stored and rotated on secrets manager. Here is how to obtain the value

module "rds_cluster" {
  source = "terraform-aws-modules/rds-aurora/aws"
  ...
  master_username = "masteruser"
  manage_master_user_password_rotation              = true
  master_user_password_rotation_schedule_expression = "rate(15 days)"
}

data "aws_secretsmanager_secret_version" "db_master_creds" {
  secret_id = module.rds_cluster.cluster_master_user_secret[0]["secret_arn"]
}

output "db_master_username" {
  value = jsondecode(data.aws_secretsmanager_secret_version.db_master_creds.secret_string)["username"]
  sensitive = true
}

output "db_master_password" {
  value = jsondecode(data.aws_secretsmanager_secret_version.db_master_creds.secret_string)["password"]
  sensitive = true
}

Keep in mind that the secret stores both username and password as a json string, so this is why you have to use jsondecode

Foveola