azure terraform: how to attach multiple data disks to multiple VMs

Question

I'm following Neal Shah's instructions for deploying multiple VMs with multiple managed disks (https://www.nealshah.dev/posts/2020/05/terraform-for-azure-deploying-multiple-vms-with-multiple-managed-disks/#deploying-multiple-vms-with-multiple-datadisks)

everything works fine except for the azurerm_virtual_machine_data_disk_attachment resource which fails with the following error

│ Error: Invalid index
│
│   on main.tf line 103, in resource "azurerm_virtual_machine_data_disk_attachment" "managed_disk_attach":
│  103:   virtual_machine_id = azurerm_linux_virtual_machine.vms[element(split("_", each.key), 1)].id
│     ├────────────────
│     │ azurerm_linux_virtual_machine.vms is tuple with 3 elements
│     │ each.key is "datadisk_dca0-apache-cassandra-node0_disk00"
│
│ The given key does not identify an element in this collection value: a number is required.

my code is below:

locals {
  vm_datadiskdisk_count_map = { for k in toset(var.nodes) : k => var.data_disk_count }
  luns = { for k in local.datadisk_lun_map : k.datadisk_name => k.lun }
  datadisk_lun_map = flatten([
    for vm_name, count in local.vm_datadiskdisk_count_map : [
      for i in range(count) : {
        datadisk_name = format("datadisk_%s_disk%02d", vm_name, i)
        lun           = i
      }
    ]
  ])
}

# create resource group
resource "azurerm_resource_group" "resource_group" { 
  name = format("%s-%s", var.dca, var.name)
  location = var.location 
}

# create availability set
resource "azurerm_availability_set" "vm_availability_set" {
  name = format("%s-%s-availability-set", var.dca, var.name)
  location = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name
}

# create Security Group to access linux
resource "azurerm_network_security_group" "linux_vm_nsg" {
  name = format("%s-%s-linux-vm-nsg", var.dca, var.name)
  location = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name
  security_rule {
    name = "AllowSSH"
    description = "Allow SSH"
    priority = 100
    direction = "Inbound"
    access = "Allow"
    protocol = "Tcp"
    source_port_range = "*"
    destination_port_range = "22"
    source_address_prefix = "*"
    destination_address_prefix = "*"
  }
}

# associate the linux NSG with the subnet
resource "azurerm_subnet_network_security_group_association" "linux_vm_nsg_association" {
  subnet_id = "${data.azurerm_subnet.subnet.id}"
  network_security_group_id = azurerm_network_security_group.linux_vm_nsg.id
}

# create NICs for apache cassandra hosts
resource "azurerm_network_interface" "vm_nics" {
  depends_on = [azurerm_subnet_network_security_group_association.linux_vm_nsg_association]
  count = length(var.nodes)
  name = format("%s-%s-nic${count.index}", var.dca, var.name)
  location = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name 
  ip_configuration { 
    name = format("%s-%s-apache-cassandra-ip", var.dca, var.name)
    subnet_id = "${data.azurerm_subnet.subnet.id}" 
    private_ip_address_allocation = "Dynamic"
  } 
} 

# create apache cassandra VMs
resource "azurerm_linux_virtual_machine" "vms" {
  count = length(var.nodes)
  name = element(var.nodes, count.index)  
  location = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name
  network_interface_ids = [element(azurerm_network_interface.vm_nics.*.id, count.index)]
  availability_set_id = azurerm_availability_set.vm_availability_set.id
  size = var.vm_size 
  admin_username = var.admin_username 
  disable_password_authentication = true
  admin_ssh_key {
    username = var.admin_username
    public_key = var.ssh_pub_key
  } 
  source_image_id = var.source_image_id
  os_disk {
    caching = "ReadWrite" 
    storage_account_type = var.storage_account_type 
    disk_size_gb = var.os_disk_size_gb
  } 
} 

# create data disk(s) for VMs
resource "azurerm_managed_disk" "managed_disk" {
  for_each= toset([for j in local.datadisk_lun_map : j.datadisk_name])
  name= each.key
  location = azurerm_resource_group.resource_group.location
  resource_group_name  = azurerm_resource_group.resource_group.name
  storage_account_type = var.storage_account_type
  create_option = "Empty"
  disk_size_gb = var.disk_size_gb
}

resource "azurerm_virtual_machine_data_disk_attachment" "managed_disk_attach" {
  for_each           = toset([for j in local.datadisk_lun_map : j.datadisk_name])
  managed_disk_id    = azurerm_managed_disk.managed_disk[each.key].id
  virtual_machine_id = azurerm_linux_virtual_machine.vms[element(split("_", each.key), 1)].id
  lun                = lookup(local.luns, each.key)
  caching            = "ReadWrite"
}

anyone know how to accomplish this? thanks!

I've tried several different approaches to this but have been unsuccessful so far, I was expecting it to work as described in Neal's post

You have to provide values of all your variables and locals, such as var.nodes. — Marcin
– Marcin, Commented Oct 26, 2022 at 23:55

RobA · Accepted Answer · 2022-10-28 14:33:34Z

I was able to get this working. However, I have not tested adding/removing nodes/disks yet. But this working to create multiple VMs with multiple data disks attached to each VM.

I use a variable file that I source to substitute the variables in the *.tf files.

variables.tf

variable "azure_subscription_id" {
  type        = string
  description = "Azure Subscription ID"
  default = ""
}
variable "dca" {
  type = string
  description = "datacenter [dca0|dca2|dca4|dca6]."
  default = ""
}

variable "location" {
  type = string
  description = "Location of the resource group."
  default = ""
}

variable "resource_group" {
  type = string
  description = "resource group name."
  default = ""
}

variable "subnet_name" {
  type = string
  description = "subnet name"
   default = ""
}

variable "vnet_name" {
  type = string
  description = "vnet name"
   default = ""
}

variable "vnet_rg" {
  type = string
  description = "vnet resource group"
   default = ""
}

variable "vm_size" {
  type = string
  description = "vm size"
   default = ""
}

variable "os_disk_size_gb" {
  type = string
  description = "vm os disk size gb"
   default = ""
}

variable "data_disk_size_gb" {
  type = string
  description = "vm data disk size gb"
   default = ""
}

variable "admin_username" {
  type = string
  description = "admin user name"
  default = ""
}

variable "ssh_pub_key" {
  type = string
  description = "public key for admin user"
   default = ""
}

variable "source_image_id" {
  type = string
  description = "image id"
  default = ""
 }

variable "os_disk_storage_account_type" {
  type = string
  description = ""
  default = ""
}

variable "data_disk_storage_account_type" {
  type = string
  description = ""
  default = ""
}

variable "vm_list" {
  type = map(object({
    hostname = string
  }))
  default = {
    vm0 ={
    hostname = "${dca}-${name}-node-0"
    },
    vm1 = {
    hostname = "${dca}-${name}-node-1"
    }
    vm2 = {
    hostname = "${dca}-${name}-node-2"
    }
  }
}

variable "disks_per_instance" {
  type = string
  description = ""
  default = ""
}

terraform.tfvars

# subscription
azure_subscription_id = "${azure_subscription_id}"
# name and location
resource_group = "${dca}-${name}"
location = "${location}"
dca = "${dca}"
# Network
subnet_name = "${subnet_name}"
vnet_name = "${dca}vnet"
vnet_rg = "th-${dca}-vnet"
# VM
vm_size = "${vm_size}"
os_disk_size_gb = "${os_disk_size_gb}"
os_disk_storage_account_type = "${os_disk_storage_account_type}"
source_image_id = "${source_image_id}" 
# User/key info
admin_username = "${admin_username}"
ssh_pub_key = ${ssh_pub_key}
# data disk info
data_disk_storage_account_type = "${data_disk_storage_account_type}"
data_disk_size_gb = "${data_disk_size_gb}"
disks_per_instance= "${disks_per_instance}"

main.tf

# set locals for multi data disks
locals {
  vm_datadiskdisk_count_map = { for k, query in var.vm_list : k => var.disks_per_instance }
  luns                      = { for k in local.datadisk_lun_map : k.datadisk_name => k.lun }
  datadisk_lun_map = flatten([
    for vm_name, count in local.vm_datadiskdisk_count_map : [
      for i in range(count) : {
        datadisk_name = format("datadisk_%s_disk%02d", vm_name, i)
        lun           = i
      }
    ]
  ])
}

# create resource group
resource "azurerm_resource_group" "resource_group" { 
  name = format("%s", var.resource_group)
  location = var.location 
}

# create data disk(s)
resource "azurerm_managed_disk" "managed_disk" {
  for_each             = toset([for j in local.datadisk_lun_map : j.datadisk_name])
  name                 = each.key
  location             = azurerm_resource_group.resource_group.location
  resource_group_name  = azurerm_resource_group.resource_group.name
  storage_account_type = var.data_disk_storage_account_type
  create_option        = "Empty"
  disk_size_gb         = var.data_disk_size_gb
}

# create availability set
resource "azurerm_availability_set" "vm_availability_set" {
  name = format("%s-availability-set", var.resource_group)
  location = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name
}

# create Security Group to access linux
resource "azurerm_network_security_group" "linux_vm_nsg" {
  name = format("%s-linux-vm-nsg", var.resource_group)
  location = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name
  security_rule {
    name = "AllowSSH"
    description = "Allow SSH"
    priority = 100
    direction = "Inbound"
    access = "Allow"
    protocol = "Tcp"
    source_port_range = "*"
    destination_port_range = "22"
    source_address_prefix = "*"
    destination_address_prefix = "*"
  }
}

# associate the linux NSG with the subnet
resource "azurerm_subnet_network_security_group_association" "linux_vm_nsg_association" {
  subnet_id = "${data.azurerm_subnet.subnet.id}"
  network_security_group_id = azurerm_network_security_group.linux_vm_nsg.id
}

# create NICs for vms
resource "azurerm_network_interface" "nics" {
  depends_on = [azurerm_subnet_network_security_group_association.linux_vm_nsg_association] 
  for_each = var.vm_list
  name = "${each.value.hostname}-nic"
  location = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name 
  ip_configuration { 
    name = format("%s-proxy-ip", var.resource_group)
    subnet_id = "${data.azurerm_subnet.subnet.id}" 
    private_ip_address_allocation = "Dynamic"
  } 
} 

# create VMs
resource "azurerm_linux_virtual_machine" "vms" {
  for_each = var.vm_list
  name = each.value.hostname
  location = azurerm_resource_group.resource_group.location
  resource_group_name = azurerm_resource_group.resource_group.name
  network_interface_ids = [azurerm_network_interface.nics[each.key].id]
  availability_set_id = azurerm_availability_set.vm_availability_set.id
  size = var.vm_size
  source_image_id = var.source_image_id
  custom_data = filebase64("cloud-init.sh") 
  admin_username = var.admin_username 
  disable_password_authentication = true
  admin_ssh_key {
    username = var.admin_username
    public_key = var.ssh_pub_key
  } 
  os_disk {
    caching = "ReadWrite" 
    storage_account_type = var.os_disk_storage_account_type 
    disk_size_gb = var.os_disk_size_gb
  } 
} 

# attache data disks vms
resource "azurerm_virtual_machine_data_disk_attachment" "managed_disk_attach" {
  for_each           = toset([for j in local.datadisk_lun_map : j.datadisk_name])
  managed_disk_id    = azurerm_managed_disk.managed_disk[each.key].id
  virtual_machine_id = azurerm_linux_virtual_machine.vms[element(split("_", each.key), 1)].id
  lun                = lookup(local.luns, each.key)
  caching            = "ReadWrite"
}

Collectives™ on Stack Overflow

azure terraform: how to attach multiple data disks to multiple VMs

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related