Terraform keeps changing line endings of the multi-line heredoc depending on the runtime environment

Question

I have this terraform resource (a bit simplified for clarity):

resource "azurerm_key_vault_secret" "env_secrets" {
  name         = "my-secret"
  key_vault_id = var.key_vault_id

  value = <<-EOT
  {
    "ADMIN_USER": "admin",
    "ADMIN_PASSWORD": "some_secret",
  }
  EOT

  content_type = "application/x-json"
}

What happens is that depending on where the terraform is run (on WSL2 on Windows or on Ubuntu in the deploy pipeline) the line ending change back and forth from \n to \r\n meaning there is all the time a "change" that should be applied which is not ideal.

Is there any good way to fix it? I assume perhaps a hard conversion to \n, or removal of \r or something like that. Maybe there are some standard ways of fixing this?

P.S. I assume that different line-endings happen because of git, but seems like the correct way on how git behaves so it should probably be fixed in terraform.

Charlean

Answer 1

That's what I ended up doing:

locals {
  value_raw = <<-EOT
  {
    "ADMIN_USER": "admin",
    "ADMIN_PASSWORD": "some_secret",
  }
  EOT
  value = chomp(replace(local.value_raw, "\r\n", "\n"))
}

resource "azurerm_key_vault_secret" "env_secrets" {

  value = local.value
}

Charlean

Answer 2

It seems that you have your version control system configured to automatically rewrite the line endings of files when extracting them on different systems. Terraform is trying to preserve your string exactly as written (because line endings can often be important for systems you're using Terraform to manage, even though that's not true for JSON) and so if you want consistent behavior on all platforms then you should generall configure your version control system to preserve exactly the bytes written in your .tf files, and not to automatically overwrite them on checkout.

If your source code is in Git then you can configure that behavior using a .gitattributes file in your repository.

---

For your particular case though, since you are generating JSON it's weird to use a heredoc string because Terraform has a jsonencode function dedicated to exactly this purpose:

resource "azurerm_key_vault_secret" "env_secrets" {
  value = jsonencode({
    "ADMIN_USER" = "admin"
    "ADMIN_PASSWORD" = "some_secret"
  })
}

Because the jsonencode function knows that it's generating JSON and therefore the whitespace is immaterial there, it'll generate a consistent result regardless of the line endings used in your source file. This approach also typically works better if you later end up wanting to generate parts of your JSON data structure dynamically, because you can use arbitrary Terraform expressions in that jsonencode argument.

Merited

Thanks a lot for help! Json values were only one of the examples, and I unfortunately have other non-json values generated the same way. It is also a problem to set git to ignore lineending as opening files on different systems would probably override line endings back and forth... I will try to switch to json where it is possible, it seems like a reasonable thing to do – Charlean

Setting conventions within your team about how to create files regardless of platform is something you'll need to coordinate outside of Terraform. For example, you could use automatic checks in your version control system to verify that new pull requests have only the expected line endings, and return an error if not. If everyone on your team is using an editor that supports it, you could also help folks generate suitable files by default by using EditorConfig. – Merited