Azure ARM template for Service bus with Topics with many Subscriptions

Question

Hi I have an ARM template for create a ServiceBus with its topics and subscriptions. But I can only accomplish 1 topic - 1 subscription because I cannot make a nested loop to create many subscriptions per topic.

I wish I could execute a template like this:

params:

{
   "serviceBusName": "mybus",
   "topics": 
    [ 
      { 
         "topicName": "mytopic1",
         "subscriptions": [ "mysubscription1", "mysubscription2"]
      },
      { 
         "topicName": "mytopic2",
         "subscriptions": [ "mysubscription1"]
      }  
    ]
}

This is my actual template:

{
  "$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "ServiceBusNamespaceName": {
      "type": "string"
    },
    "ServiceBusSku": {
      "type": "string",
      "allowedValues": [
        "Basic",
        "Standard"
      ],
      "defaultValue": "Standard"
    },
    "ServiceBusSmallSizeTopicInMb": {
      "type": "int",
      "defaultValue": 1024
    },
    "ServiceBusMaxSizeTopicInMb": {
      "type": "int",
      "defaultValue": 1024
    },
    "Topics": {
      "type": "array"
    }
  },
  "variables": {
    "DefaultSASKeyName": "RootManageSharedAccessKey",
    "DefaultAuthRuleResourceId": "[resourceId('Microsoft.ServiceBus/namespaces/authorizationRules', parameters('ServiceBusNamespaceName'), variables('DefaultSASKeyName'))]",
    "SbVersion": "2017-04-01"
  },
  "resources": [
    {
      "apiVersion": "2017-04-01",
      "name": "[parameters('ServiceBusNamespaceName')]",
      "type": "Microsoft.ServiceBus/namespaces",
      "location": "[resourceGroup().location]",
      "sku": {
        "name": "[parameters('ServiceBusSku')]"
      },
      "tags": {
        "displayName": "ServiceBus"
      }
    },
    {
      "copy": {
        "name": "topics",
        "count": "[length(parameters('Topics'))]"
      },
      "type": "Microsoft.ServiceBus/namespaces/topics",
      "name": "[concat(parameters('ServiceBusNamespaceName'), '/', parameters('Topics')[copyIndex()].topic)]",
      "apiVersion": "2017-04-01",
      "location": "[resourceGroup().location]",
      "scale": null,
      "properties": {
        "defaultMessageTimeToLive": "P1D",
        "maxSizeInMegabytes": "[parameters('ServiceBusMaxSizeTopicInMb')]",
        "requiresDuplicateDetection": false,
        "duplicateDetectionHistoryTimeWindow": "PT10M",
        "enableBatchedOperations": true,
        "status": "Active",
        "supportOrdering": true,
        "autoDeleteOnIdle": "P10675199DT2H48M5.4775807S",
        "enablePartitioning": false,
        "enableExpress": false
      },
      "dependsOn": [
        "[resourceId('Microsoft.ServiceBus/namespaces', parameters('ServiceBusNamespaceName'))]"
      ],
      "resources": [
        {
          "apiVersion": "[variables('sbVersion')]",
          "name": "[parameters('Topics')[copyIndex()].subscription]",
          "type": "Subscriptions",
          "dependsOn": [
            "[parameters('Topics')[copyIndex()].topic]"
          ],
          "properties": {
            "lockDuration": "PT1M",
            "requiresSession": "false",
            "defaultMessageTimeToLive": "P7D",
            "deadLetteringOnMessageExpiration": "false",
            "maxDeliveryCount": "2",
            "enableBatchedOperations": "true",
            "autoDeleteOnIdle": "P7D"
          }
        }
      ]
    }
  ],
  "outputs": {
    "NamespaceDefaultConnectionString": {
      "type": "string",
      "value": "[listkeys(variables('DefaultAuthRuleResourceId'), variables('SbVersion')).primaryConnectionString]"
    },
    "DefaultSharedAccessPolicyPrimaryKey": {
      "type": "string",
      "value": "[listkeys(variables('DefaultAuthRuleResourceId'), variables('SbVersion')).primaryKey]"
    }
  }
}

And an example of the params json for the actual template:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "ServiceBusNamespaceName": {
            "value": "mybus"
        },
        "ServiceBusSku": {
            "value": "Standard"
        },
        "Topics ": {
            "value": [
                {
                    "topic": "mytopic1",
                    "subscription": "mysubscription1"
                },
                {
                    "topic": "mytopic2",
                    "subscription": "mysubscription1"
                }
            ]
        }
    }
}

Connate

Answer 1

In general, there are two ways to do something like this. You can restructure your code so that the subscriptions are top level resources. Or you use the named variant of copyIndex to achieve nested loops. Both variants can be seen in this blog post and the comment for it.

Azure ARM Templates – Are nested loops possible?

However, for your case, the only option is to make the subscriptions a top level resource. See aka.ms/arm-copy/#looping-on-a-nested-resource for more details.

This is the full example taken from the blog post mentioned above.

{
    "$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "serviceBusNamespaceName": {
            "type": "string",
            "metadata": {
                "description": "Name of the Service Bus namespace"
            }
        },
        "topics":{
            "type": "array",
            "metadata": {
                "description": "List of topics"
            }
        },
        "subscriptions":{
            "type": "array",
            "metadata": {
                "description": "List of subscriptions"
            }
        }

    },
    "variables": {},
    "resources": [
        {
            "type": "Microsoft.ServiceBus/namespaces",
            "sku": {
                "name": "Standard"
            },
            "name": "[parameters('serviceBusNamespaceName')]",
            "apiVersion": "2017-04-01",
            "location": "[resourceGroup().location]",
            "properties": {}
        },
        {
            "type": "Microsoft.ServiceBus/namespaces/topics",
            "name": "[concat(parameters('serviceBusNamespaceName'), '/', parameters('topics')[copyIndex()])]",
            "apiVersion": "2017-04-01",
            "location": "[resourceGroup().location]",
            "copy": {
                "name": "topicLoop",
                "count": "[length(parameters('topics'))]"
            },
            "properties": {},
            "dependsOn": [
                "[concat('Microsoft.ServiceBus/namespaces/', parameters('serviceBusNamespaceName'))]"
            ]
        },
        {
            "type": "Microsoft.ServiceBus/namespaces/topics/subscriptions",
            "name": "[concat(parameters('serviceBusNamespaceName'), '/', parameters('subscriptions')[copyIndex()].topic, '/', parameters('subscriptions')[copyIndex()].subscription)]",
            "apiVersion": "2017-04-01",
            "location": "[resourceGroup().location]",
            "copy": {
                "name": "subscriptionLoop",
                "count": "[length(parameters('subscriptions'))]"
            },
            "properties": {},
            "dependsOn": [
                "topicLoop"
            ]
        }
    ]
}

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
      "serviceBusNamespaceName": {
        "value": "rjtestsbnmspace"
      },
      "topics": {
        "value": ["topic1", "topic2"]
      },
      "subscriptions": {
        "value": [{
          "topic": "topic1",
          "subscription": "subscription1"
          },
          {
            "topic": "topic1",
            "subscription": "subscription2"
          },
          {
            "topic": "topic2",
            "subscription": "subscription3"
          }
        ]
      }
    }
  }

This example for VMs uses named copyIndex - it works when the nested loop is not for a resource itself.

{
  "name": "[concat(parameters('vmName'), padLeft(copyIndex(1), 2, '0'))]",
  "type": "Microsoft.Compute/virtualMachines",
  "copy": {
    "name": "vmLoop",
    "count": "[parameters('vmCount')]"
  },
  "properties": {
    "osProfile": {
      "computerName": "[concat(parameters('vmName'), padLeft(copyIndex(1), 2, '0'))]"
    },
    "hardwareProfile": {
      "vmSize": "[parameters('vmSize')]"
    },
    "storageProfile": {
      "osDisk": {
        "name": "[concat(parameters('vmName'), padLeft(copyIndex(1), 2, '0'),'_OSDisk')]",
        "createOption": "FromImage",
        "managedDisk": {
          "storageAccountType": "[parameters('vmDiskType')]"
        }
      },
      "copy": [
        {
          "name": "dataDisks",
          "count": "[parameters('dataDiskCount')]",
          "input": {
            "caching": "[parameters('dataDiskCaching')]",
            "name": "[concat(parameters('vmName'), padLeft(copyIndex('vmLoop', 1), 2, '0'), '-dataDisk', padLeft(copyIndex('dataDisks'), 2, '0'))]",
            "lun": "[copyIndex('dataDisks')]",
            "createOption": "Empty",
            "diskSizeGB": "[parameters('dataDiskSize')]",
            "managedDisk": {
              "storageAccountType": "[parameters('vmDiskType')]"
            }
          }
        }
      ]
    }
  }
}

Predigestion

Great answer, thanks. I'm afraid the nested loops with named copyIndex doesn't really apply in the service bus topics/subscriptions case, as they're both resources. I tried it but got the error: "Copying nested resources is not supported. Please see aka.ms/arm-copy/#looping-on-a-nested-resource for usage details" – Arlyn

Ok, in this case it seems you have to fall back to the first version I mentioned. I will edit the answer. Thank you for the comment – Predigestion

Answer 2

you can achieve that using nested deployment, high level view would be like this:

{
    "apiVersion": "2017-05-10",
    "name": "[concat('topicLoop-', copyIndex())]",
    "type": "Microsoft.Resources/deployments",
    "copy": {
        "name": "topicLoop",
        "count": "[length(parameters('topics'))]"
    },
    "dependsOn": [
       // this should on the service bus resource
    ]
    "properties": {
        "mode": "Incremental",
        "templateLink": {
            "uri": "nested_template_link"
        },
        "parameters": {
            "iteration": {
                "value": "[parameters('topics')[copyIndex()]]"
            }
        }
    }
},

and then inside your nested template you would have something like this:

{
    "type": "Microsoft.ServiceBus/namespaces/topics",
    "name": "[concat(parameters('serviceBusNamespaceName'), '/', parameters('iteration').topicName)]",
    "apiVersion": "2017-04-01",
    "location": "[resourceGroup().location]"
},
{
    "type": "Microsoft.ServiceBus/namespaces/topics/subscriptions",
    "name": "[concat(parameters('serviceBusNamespaceName'), '/', parameters('iteration').topicName, '/', parameters('iteration').subscriptions[copyIndex()])]",
    "apiVersion": "2017-04-01",
    "location": "[resourceGroup().location]",
    "copy": {
        "name": "subscriptionLoop",
        "count": "[length(parameters('iteration').subscriptions)]"
    },
    "properties": {},
    "dependsOn": [
        "[concat(parameters('serviceBusNamespaceName'), '/', parameters('iteration').topicName)"]
    ]
}

this would allow you to keep existing parameter structure and make the template more flexible, the other answers solution is not really maintainable (you'd have to edit 2 parameters, instead of just one, which is a big deal, tbh)

Reexamine

Answer 3

I have managed to get it working while using the most simple topics/subscription structure and using a nested loop with sub-deployments. An important factor in this is that you use an internal scope for the sub deployment, and pass the topic you want to create the subscriptions for as parameter to the sub-deployment.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {

        // Since we don't want to create complex ARM structure, we use the most basic variable structure
        // to define the topics and subscriptions.
        // The complexity is in the ARM template.
        "topics": {
            "type": "array",
            "defaultValue":
            [
                {
                    "name": "articles",
                    "subscriptions": [
                        "diagnostics",
                        "orders"
                    ]
                },
                {
                    "name": "customers",
                    "subscriptions": [
                        "diagnostics",
                        "orders"
                    ]
                },
                {
                    "name": "orders",
                    "subscriptions": [
                        "diagnostics",
                        "orders"
                    ]
                }
            ]
        }
    },
    "variables": {
        "baseName": "[resourceGroup().name]",
        "serviceBusName": "[variables('baseName')]",
        "topics": "[parameters('topics')]"
    },
    "resources": [
        {
            //
            // Create the Service-bus itself.
            //
            "name": "[variables('serviceBusName')]",
            "type": "Microsoft.ServiceBus/namespaces",
            "apiVersion": "2018-01-01-preview",
            "location": "[resourceGroup().location]",
            "sku": { "name": "Standard" },
            "kind": "Messaging",
            "properties": {},
            "resources": [
            ]
        },
        {
            //
            // Now we are going to create the topics. This is a loop throught the topics variable.
            //
            "copy": {
                "name": "topics-loop",
                "count": "[length(variables('topics'))]"
            },
            "name": "[concat(variables('serviceBusName'),'/', variables('topics')[copyIndex()].name)]",
            "type": "Microsoft.ServiceBus/namespaces/topics",
            "apiVersion": "2017-04-01",
            "dependsOn": [
                "[concat('Microsoft.ServiceBus/Namespaces/', variables('serviceBusName'))]"
            ],
            "properties": {}
        },
        {
            //
            // The following structure looks rather complex. Since nested loops are not supported for resources (it is for properties),
            // we create a deployment that is looped and passes the required variables as parameters to the inner deployment.
            // In the inner deployment another loop is created, and this creates the subscriptions.
            //

            "copy": {
                "name": "subscriptions-outer-loop",
                "count": "[length(variables('topics'))]"
            },

            // The name of the deployment.
            "name": "[concat(variables('serviceBusName'),'-',variables('topics')[copyIndex()].name,'-subscriptions')]",
            "type": "Microsoft.Resources/deployments",
            "apiVersion": "2019-10-01",
            "properties": {
                "mode": "Incremental",

                // Set the scope to Inner. Everything from the outer deployment that is needed in the inner template,
                // can be moved via parameters. You cannot do nested loops if you let te scope be outer.
                "expressionEvaluationOptions": {
                    "scope": "inner"
                },

                // The following parameters are defined by the inner template.
                "parameters": {
                    "serviceBusName": {
                        "value": "[variables('serviceBusName')]"
                    },

                    // The current topic, this is an important one, 
                    // it communicates the current topic to the inner template
                    "currentTopic": {
                        "value": "[variables('topics')[copyIndex()]]"
                    }
                },
                "template": {
                    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
                    "contentVersion": "1.0.0.0",
                    "parameters": {
                        // Define the parameters that are set by the outer template.
                        "serviceBusName": { "type": "string" },
                        "currentTopic": { "type": "object" }
                    },
                    "resources": [
                        {
                            // The inner loop. This will create the subscriptions.
                            "copy": {
                                "name": "subscriptions-inner-loop",
                                "count": "[length(parameters('currentTopic').subscriptions)]"
                            },
                            "name": "[concat(
                                    parameters('serviceBusName'),'/', 
                                    parameters('currentTopic').name, '/',
                                    parameters('currentTopic').subscriptions[copyIndex()])]",
                            "type": "Microsoft.ServiceBus/namespaces/topics/subscriptions",
                            "apiVersion": "2017-04-01",
                            "properties": {}
                        }
                    ]
                }
            },

            //This depends on the outer loop.
            "dependsOn": [ "topics-loop" ]
        }
    ]
}

Jezabel

Thank you so much for this idea! I tuned it a little bit and made it work under Azure DevOps to deploy the resources from a file in the git repo. I need to add the logic for supporting filters on the subscriptions and queues. I will dive in with this but some help would be wonderful. :) – Acidulant