Nested Loops in Terraform

When I first wanted to do nested looping in terraform, a lot of information online said it wasn’t possible. I learned eventually that it is possible, with one caveat, and this is the post I wish I’d found back then.

Typical looping syntax in terraform

Imagine we’ve got a map of items:

locals {
	items = {
		"value1": {...},
		"value2": {...},
		...
		"valueN": {...},
	}
}

The way you loop over these items in a flat fashion is like this with a for_each expression

for_each = local.items

## each.key is `valueX`
## each.value is the object at that key

I’ll use the example of DataDog dashboard widgets throughout, to keep the demos consistent. With this loop, we could define some graph templates and generate them with a for_each.

resource "datadog_dashboard" "new_dash" {
	for_each = local.items
	widget {
	  ...
	  query = each.value.query
	  ...
	}
}

No nesting

The standard for_each approach does not support nesting though.

Imagine this use case: for a list of services, I want to generate a set of graphs monitoring specific kinds of error produced by that service, grouped by their service.

If I were writing that in Go it would look like:

for _, service := range service {
  serviceGroup := datadog.NewGroup()

  for _, graph := range service.graphs {
    serviceGroup.Add(graph)
  }
}

But the same syntax doesn’t work in Terraform. A second for_each statement won’t work that way.

NB: IF YOU’RE JUST SCANNING THIS POST, THE FOLLOWING APPROACH DOES NOT WORK. KEEP READING.

locals {
  services = {
     service1": {
       "serviceName": "...",
       "errorNames": [....]
     },
     "service2": {
       "serviceName": "...",
       "errorNames": [....]
     },
   }
}

resource "datadog_dashboard" "new_dash" {
  for_each = local.services
  widget {

    # a group of graphs under a heading
    group_definition {
      title = "each.value.serviceName"

      #
      # it will break here
      #
      for_each = each.value.errorNames
      widget {...}
    }
  }
}

You’d imagine we could just nest for_each statements like this, but we can’t.

Flattening option

One option is to use the flatten(...) method, to essentially un-nest the values and create a flat list with one element per thing you want to create.

This would work fine if you don’t care about grouping the results, but in our case and in many use cases, the nesting represents a relationship between the sub-elements which you want to retain.

Nesting with “dynamic” blocks

The solution is to use dynamic blocks. These let you generate one block per iterated value, and crucially allows you to nest these iterators.

Here’s how it works with the same example:

locals {
  services = {
     service1": {
       "serviceName": "...",
       "errorNames": [....]
     },
     "service2": {
       "serviceName": "...",
       "errorNames": [....]
     },
   }
}


resource "datadog_dashboard" "new_dash" {
	
  # add dynamic in front of the block to repeat
  # this gives us one widget per service
  dynamic "widget" {

    # loop over services
    for_each = local.services

    # and name the iterator variable
    iterator = service

    # content keyword separates the iteration syntax and content
    content {
      group_definition {
        title = "each.value.serviceName"

        # another dynamic
        # one widget per error_name
        dynamic "widget" {
          # refer to the iterator variable "service"
          # and loop over its errors
          for_each = service.value.errorNames
          iterator = "error_name"

          content {...}
        }
      }
    }
  }
}

That’s how we do nested looping in Terraform. A few things to note:

• The dynamic keyword is key. That’s the block which will repeat, for each iterator value inside it.
• The content keyword separates the loop syntax from what goes inside the repeating block. Make sure you put everything inside a content block for dynamic widgets.
• The iterator value allows you to name the variable which holds the item. Do NOT put quotes around your iterator variable name, e.g. iterator = "service", which I found will serve cryptic errors.

Caveat: Only blocks can repeat

Because this is a dynamic block feature, we can only iterate on blocks. You can’t place loops wherever you wish. They have to go inside the dynamic block.

You can’t loop over services, then over errorNames without them each corresponding to a block. If you want to do that though, flattening is probably fine for your use case.