How to use multiple AWS account to isolate terraform state between environment

Question

How can I do to use s3 backend that points to a different AWS account?

In other words, I would like to have something like that:

Dev environment state on an S3 bucket in AWS account A

Stage environment state on another S3 bucket on AWS account B

Anyone can help me, please?

Luciana

Answer 1

The documentation for Terraform's s3 backend includes a section Multi-account AWS Architecture which includes some recommendations, suggestions, and caveats for using Terraform in a multi-account AWS architecture.

That guide is far more detailed than I can reproduce here, but the key points of recommendation are:

• Use a separate AWS account for Terraform and any other administrative tools you use to provision and configure your environments, so that the infrastructure that Terraform uses is entirely separate from the infrastructure that Terraform manages.

  This reduces the risk of an incorrect Terraform configuration inadvertently breaking your ability to use Terraform itself (e.g. by deleting the state object, or by removing necessary IAM permissions). It also reduces the possibility for an attacker to use vulnerabilities in your main infrastructure to escalate to access to your administrative infrastructure.

• Use sts:AssumeRole to indirectly access IAM roles with administrative access in each of your main environment AWS accounts.

  This allows you to centralize all of your direct administrative access in a single AWS account where you can more easily audit it, reduces credentials sprawl, and also conveniently configure the AWS provider for that cross-account access (because it has assume_role support built-in).

The guide also discusses using workspaces to represent environments. That advice is perhaps more debatable given the guidance elsewhere in When to use Multiple Workspaces, but the principle of using an administrative account and IAM delegation is still applicable even if you follow this advice of having a separate root module per environment and using shared modules to represent common elements.

As with all things in system architecture, these aren't absolutes and what is best for your case will depend on your details, but hopefully the content in these two documentation sections I've linked to will help you weigh various options and decide what is best for your specific situation.

Atlanta

thank you for your reply. I'm reading documentation, especially links that you suggest. I would like to have an s3 bucket for state management of dev environment on AWS Account A and another s3 bucket on AWS Account B to manage of prod environment (for example) I'm starting to know that the only way to do that with terraform is to duplicate infrastructure code for every environment. Maybe it can be done using terragrunt. What do you think about that? – Luciana

Answer 2

There are a few solutions to it:

1. provide aws profile name at the command line while running terraform init and injec terraform backend variables during runtime:

   AWS_PROFILE=aws-dev terraform init -backend-config="bucket=825df6bc4eef-state" \
   -backend-config="dynamodb_table=825df6bc4eef-state-lock" \
   -backend-config="key=terraform-multi-account/terraform.tfstate"
   

or wrap this command in a Makefile as it is pretty long and forgettable.

2. Keep separate directories and provide the roles or your credentials or profile name even using shared-credentials

    provider "aws" {
    region                  = "us-west-2"
    shared_credentials_file = "/Users/tf_user/.aws/creds"
    profile                 = "customprofile"
    }
   

3. Terraform Workspaces

4. terragrunt

Kilovoltampere

Thank you @Kilovoltampere to show me all options. Until now i have tried Always First way, but i would like to try with third option. In this case i don't understand How can i do relation between workspace and backend configuration. – Luciana

I cant describe the whole functionality, this might get you start manage-multiple-environments-with-terraform-worksp – Kilovoltampere

it's a great link but speak about how to manage multi environment using workspace. I need a way to associate an environment with a workspace and a specific aws account. I need to get indipendent s3 backend management state between environment – Luciana

yes, you can do that by configuring the backend for each of environment with different state files. – Kilovoltampere

Answer 3

I don't think it is possible to have a separate S3 backend for each workspace without some hijinks at this time. If you are ok with one S3 backend in one account it's pretty easy to have different accounts associated with each workspace.

# backend.tf
terraform {
  backend "s3" {
    profile        = "default"
    bucket         = "my-terraform-state"
    key            = "terraform-multi-account-test/terraform.state"
    region         = "eu-west-1"
    encrypt        = true
    dynamodb_table = "my-terraform-state-lock"
  }
}

and

# provider.tf
variable "workspace_accounts" {
  type = map(string)
  default = {
    "sandbox" = "my-sandbox-keys"
    "dev"     = "default"
    "prod"    = "default"
  }
}

provider "aws" {
  shared_credentials_file = "$HOME/.aws/credentials"
  profile                 = var.workspace_accounts[terraform.workspace]
  region                  = "eu-west-1"
}

See https://github.com/hashicorp/terraform/issues/16627

Kassa