Terraform 0.12 nested for loops

Question

I am trying to implement nested for loops using Terraform 0.12's new features in order to loop through AWS IAM users, each of which can have one or more policies attached. The variable used to represent this list is of type map(list(string)) and looks something like this:

{
  "user 1" = [ "policy1", "policy2" ],
  "user 2" = [ "policy1" ]
}

Getting the list of users to create is easy enough via keys(), but since there is currently no mechanism for nesting looped resource creation in Terraform, the policy attachments have to happen as a singular loop independent of each user. So, I am attempting to construct a list of user:policy associations from the map input that would look something like this based on the example above:

[
  [ "user1", "policy1" ],
  [ "user1", "policy2" ],
  [ "user2", "policy1" ]
]

I am attempting construct that list and store it in a local variable like so, where var.iam-user-policy-map is the input map:

locals {
  ...
  association-list = [
    for user in keys(var.iam-user-policy-map):
    [
      for policy in var.iam-user-policy-map[user]:
      [user, policy]
    ]
  ]
  ...
}

However, I am getting errors when attempting to access the values in that nested list. I am trying to access the user portion of the association with the reference local.association-list[count.index][0] and the policy with local.association-list[count.index][1], but on running terraform plan it errors out:

Error: Incorrect attribute value type

  on main.tf line 27, in resource "aws_iam_user_policy_attachment" "test-attach":
  27:   user = local.association-list[count.index][0]

Inappropriate value for attribute "user": string required.


Error: Incorrect attribute value type

  on main.tf line 27, in resource "aws_iam_user_policy_attachment" "test-attach":
  27:   user = local.association-list[count.index][0]

Inappropriate value for attribute "user": string required.


Error: Invalid index

  on main.tf line 28, in resource "aws_iam_user_policy_attachment" "test-attach":
  28:   policy_arn = "arn:aws-us-gov:iam::aws:policy/${local.association-list[count.index][1]}"
    |----------------
    | count.index is 0
    | local.association-list is tuple with 2 elements

The given key does not identify an element in this collection value.


Error: Invalid template interpolation value

  on main.tf line 28, in resource "aws_iam_user_policy_attachment" "test-attach":
  28:   policy_arn = "arn:aws-us-gov:iam::aws:policy/${local.association-list[count.index][1]}"
    |----------------
    | count.index is 1
    | local.association-list is tuple with 2 elements

Cannot include the given value in a string template: string required.

What am I doing wrong?

Kneepad

Answer 1

The for expression in your local value association-list is producing a list of list of lists of strings, but your references to it are treating it as a list of lists of strings.

To get the flattened representation you wanted, you can use the flatten function, but because it would otherwise group everything into a single flat list I'd recommend making the innermost value an object instead. (That will also make the references to it clearer.)

locals {
  association-list = flatten([
    for user in keys(var.iam-user-policy-map) : [
      for policy in var.iam-user-policy-map[user] : {
        user   = user
        policy = policy
      }
    ]
  ])
}

The result of this expression will have the following shape:

[
  { user = "user1", policy = "policy1" },
  { user = "user1", policy = "policy2" },
  { user = "user2", policy = "policy2" },
]

Your references to it can then be in the following form:

user = local.association-list[count.index].user

policy_arn = "arn:aws-us-gov:iam::aws:policy/${local.association-list[count.index].policy}"

Hillier

I think this is a better design than what I had initially envisioned, thanks. – Kneepad

there is another issue here in that numerically referenced elements are affected by changes to items in the list. If you have multiple policy attachments and you change one of them, terraform will show changes to other policy attachments that you have not changed as its reordered the list. Using a for_each in the module would result in key referenced elements e.g. module.iam-policy-attachments.aws_iam_role_policy_attachment.role_policy["policy_name"] rather than module.iam-policy-attachments.aws_iam_role_policy_attachment.role_policy[1] so a single policy change would not affect others – Tagmeme

there is a good explanation here blog.gruntwork.io/… – Tagmeme

There's a very useful example in the official docs that explains how to use this and then create a map from it that you can use with for_each: terraform.io/docs/language/functions/… – Palladic

Answer 2

If you need a map for 'for_each', 'merge' is convenient.

variable "iam-user-policy-map" {
  default = {
    "user 1" = ["policy1", "policy2"],
    "user 2" = ["policy1"]
  }
}

locals {
  association-map = merge([
    for user, policies in var.iam-user-policy-map : {
      for policy in policies :
        "${user}-${policy}" => {
          "user"   = user
          "policy" = policy
        }
    }
  ]...)
}

output "association-map" {
  value = local.association-map
}

Outputs:

association-map = {
  "user 1-policy1" = {
    "policy" = "policy1"
    "user" = "user 1"
  }
  "user 1-policy2" = {
    "policy" = "policy2"
    "user" = "user 1"
  }
  "user 2-policy1" = {
    "policy" = "policy1"
    "user" = "user 2"
  }
}

Example for_each usage:

resource "null_resource" "echo" {
  for_each = local.association-map
  provisioner "local-exec" {
    command = "echo 'policy - ${each.value.policy}, user - ${each.value.user}'"
  }
}

https://github.com/hashicorp/terraform/issues/22263#issuecomment-969549347

Heterosexuality

Hi, Really apricate your answer. But I could not understand the use of three dots here. locals { association-map = merge([ for user, policies in var.iam-user-policy-map : { for policy in policies : "${user}-${policy}" => { "user" = user "policy" = policy } } ]...) } – Hooknosed

This does not seem to work anymore: Call to function "merge" failed: arguments must be maps or objects, got "tuple". – Unionism

Sorry, it works, I was underestimating the importance of .... Wow... – Unionism

this such a useful pattern for terraform !!!!!! – Ecg

what does that '...' at the end of the merge mean? seems it is necessary – Orelie

This answer is way safer than the originally accepted answer. – Exsanguinate

Answer 3

I am not sure where I got this answer from, but this one worked for me.

locals {
  schemas    = [
                 "PRIVATE",
                 "PUBLIC",
                 "MY_SCHEMA",
               ]
  privileges = [
                 "CREATE TABLE",
                 "CREATE VIEW",
                 "USAGE",
               ]
  # Nested loop over both lists, and flatten the result.
  schema_privileges = distinct(flatten([
    for schema in local.schemas : [
      for privilege in local.privileges : {
        privilege = privilege
        schema    = schema
      }
    ]
  ]))
}

 resource "snowflake_schema_grant" "write_permissions" {
  # We need a map to use for_each, so we convert our list into a map by adding a unique key:
  for_each      = { for entry in local.schema_privileges: "${entry.schema}.${entry.privilege}" => entry }
  database_name = "MY_DATABASE"
  privilege     = each.value.privilege
  roles         = "DAVE"
  schema_name   = each.value.schema
}

Colorfast

You got it from here daveperrett.com/articles/2021/08/19/… – Aymara

Could be, but I could swear it was from a SO answer. – Colorfast

It certainly could have been, but doing a google search brings up that blog post. Someone else may have also reposted it and that's what you remember. Not an accusation, I was just linking the original which I found on a similar search. – Aymara