Zero2Hero - Using Aria Automation to Deploy Multiple Machines with Multiple Disks - Part 5

Published on 29 March 2024 by Christopher Lewis. Words: 1820. Reading Time: 9 mins.

In the final part of this five post blog series, we will complete our #Zero2Hero journey by looking at how we can deploy multiple machines each with identical additional on-demand disk configurations. In this final post we will at the last stop on our journey and review the final VMware Cloud Template!

The series will cover the following high level use cases:

Creating a Single Virtual Machine Cloud Template (boot disk only) - Part 1
Creating a Single Virtual Machine Cloud Template (with additional static disk configuration) - Part 2
Creating a Multiple Virtual Machine Cloud Template (with additional static disk configuration) - Part 3
Creating a Single Virtual Machine Cloud Template (with additional on-demand disk configurations) - Part 4
Creating a Multiple Virtual Machine Cloud Template (with additional on-demand disk configurations) - Part 5

Note: All of the above use cases, and the VMware Cloud Templates described, could be used as standalone templates, but I thought it was important to show incremental development, i.e. we’re going from Zero to Hero.

Note: I did start to write this as a single blog post but the more and more I wrote the post, the more I realized it was a big topic to get right and explain and, therefore, it was best to provide it in smaller posts.

The Requirements (Recap)

As a reminder, we had the following high level requirements when developing this cloud template:

The consumer must be able to deploy one or more Virtual Machines.
The consumer must be able to choose the operating system (image) for the Virtual Machine.
The consumer must be able to choose the Virtual Machine size (flavor) at request time.
The consumer Must be able to choose to which cloud the Virtual Machine is deployed to (i.e. VMware Cloud or Private Cloud)
Each Virtual Machine must have identical disk configuration.
The disk configuration must support one to five additional disks.
The disk configuration must be chosen at request time.

The Development Process

It has been a long journey to get to here, but the final part of this article is to enhance the previous Multiple Virtual Machine Cloud Template (with additional static disks configuration) to allow the dynamic selection of N disks to X number of Virtual Machines.

So lets explore how we can do that!

Creating Multiple Virtual Machines (with additional dynamic disk configuration)

I’m going to concentrate on how to combine the VMware Cloud Template we created in Part 3 with the VMware Cloud Template we created in Part 4 .

Inputs

The majority of inputs of the VMware Cloud Template have remained the same, we continue to make use of the diskConfig array that we introduced in Part 4 of the series.

Using the same scenario as before, we have requested 2 Virtual Machines and each Virtual Machine has 2 disks. So now let us explore what happens in the VMware Cloud Template.

If, during request time, the consumer added two disks, the values within the diskConfig array would look like this:

item	controller	unit	size	drive	label
0	SCSI_Controller_1	1	100	d	apps
1	SCSI_Controller_2	2	200	e	data

If, during request time, the consumer added two disks but also wants to deploy two machines, the values within the diskConfig array would look like this:

item	controller	unit	capacity	drive	label
0	SCSI_Controller_1	1	100	d	apps
1	SCSI_Controller_2	2	200	e	data
2	SCSI_Controller_1	1	100	d	apps
3	SCSI_Controller_2	2	200	e	data

Note: There is no logic or check to stop the consumer selecting the wrong input here. The Unit Number on each SCSI Controller can only be assigned to a single disk. For example, if the consumer adds two disks on the same SCSI Controller and Unit Number then the deployment will just fail.

Resources

Cloud.vSphere.Machine

To support this multi-disk deployment we again need to update the Cloud.vSphere.Machine resource. First we are going to re-introduce the count property and the allocatePerInstance Resource Flag that we used in Part 3 . Then we are going to make a subtle change to the expression on the attachedDisks property to cater for the dynamic nature of the number of disks being requested.

attachedDisks: ${map_to_object(slice(resource.disk[*].id, length(input.diskConfig) * count.index, 
length(input.diskConfig) * (count.index +1)), "source")}

Let us try and break that down (with the example) to make it easier to understand:

count is the number of elements in the Cloud.vSphere.Machine resource array. As this is 2, we have vm[0] and vm[1].
count.index is the position of the current Virtual Machine in the resource array. As we iterate through the array, this will be 0 and then 1.
count.index+1 is just pure maths as we add 1 to the index of the array. As we iterate through the array, this will be 1 and then 2.

The above is the same as the previous VMware Cloud Template, but from now on is where it gets interesting!

resource.disk[*] is an array of Cloud.vSphere.Disks objects. As we have 2 Virtual Machines x 2 disks, then there will be 4 elements in this array, i.e. resource.disk[0], resource.disk[1], resource.disk[2] and resource.disk[3].
resource.disk[*].id is an array of the id’s for the resource.disk resource objects. As we have 4 disks, then there will be 4 resource id’s in this array, i.e. resource.disk[0].id, resource.disk[1].id, resource.disk[2].id and resource.disk[3].id.
length(input.diskConfig) is the total number of items in diskConfig array. As we are configuring 2 disks per Virtual Machine this is 2.
slice(resource.disk[*].id, length(input.diskConfig) * count.index, length(input.diskConfig) * (count.index +1)) - As we iterate through each of the Cloud.vSphere.Machine objects in the resource array, this is used to return a specific subset (or slice) of the resource array. This translates to:

On the first iteration through (for vm[0]), return an array containing a single element between index (2 x 0 =) 0 and index (2 x 1 =) 2, i.e. resource.disk[0].id and resource.disk[1].id.

On the second iteration through (for vm[1])), return an array containing a single element between index (2 x 1 =) 2 and index (2 x 2=) 4, i.e. resource.disk[2].id and resource.disk[3].id.

map_to_object(slice(resource.disk[*].id, length(input.diskConfig) * count.index, length(input.diskConfig) * (count.index +1)), "source") returns the elements of the sliced resource.disk[*].id array to the attachedDisks property using the key field called source.

Therefore if we could visualize the expanded expression syntax to see what the YAML would actually look like, we would see:

For vm[0], the first element in the resource array, this expands to:

  vm[0]:
    type: Cloud.vSphere.Machine
    properties:
      image: windows
      flavor: small
      constraints:
        - tag: cloud:vmc
      attachedDisks:
        - source: ${resource.disk[0].id}
        - source: ${resource.disk[1].id}

For vm[1], the second element in the resource array, this expands to:

  vm[1]:
    type: Cloud.vSphere.Machine
    properties:
      image: windows
      flavor: small
      constraints:
        - tag: cloud:vmc
      attachedDisks:
        - source: ${resource.disk[2].id}
        - source: ${resource.disk[3].id}

Cloud.vSphere.Disk

Whilst only a small amount changed in the Cloud.vSphere.Machine resource object from the previous VMware Cloud Template, a lot has changed in the Cloud.vSphere.Disk resource. So let us break that down for the important properties of the Cloud.vSphere.Disk object!

count is the number of elements in the Cloud.vSphere.Disk resource array.
capacityGb is the size of the new disk in Gigabytes and not Gigabits (thank you camel case).
SCSIController is the name of the SCSI Controller to attach the disk to on the Virtual Machine.
unitNumber is the Unit Number on the SCSI Controller where the disk is attached.

So how do we extract the data from the diskConfig input array? Let us break that down:

count: ${input.numVms * length(input.diskConfig)} is a formula the number of selected Virtual Machines (2) times the number of disks per Virtual Machine (2). Therefore 2 x 2 = 4.
capacityGb: ${input.diskConfig[count.index % length(input.diskConfig)].size} - here we are doing some harder maths than we have done before. What we are trying to get to is ${input.diskConfig[x].size} but as we have an array of 4 objects we need to choose the right one. We do this by doing the following maths:

Note: The % is actually a representation of the Modulus operations which means it returns the remainder of the division of two values - is your head hurting yet?

On the first iteration, count.index = 0, length(input.diskConfig) = 4. Therefore 0 mod 4 = 0 r0 which means ${input.diskConfig[0].size} is used.

On the second iteration, count.index = 1, length(input.diskConfig) = 4. Therefore 1 mod 4 = 0 r1 which means ${input.diskConfig[1].size} is used.

On the third iteration, count.index = 2, length(input.diskConfig) = 4. Therefore 2 mod 4 = 0 r2 which means ${input.diskConfig[2].size} is used.

On the fourth iteration, count.index = 3, length(input.diskConfig) = 4. Therefore 3 mod 4 = 0 r3 which means ${input.diskConfig[3].size} is used.

We use the same process to get the other values out of the input objects, including:

SCSIController: ${input.diskConfig[count.index % length(input.diskConfig)].controller}
unitNumber: ${to_string(input.diskConfig[count.index % length(input.diskConfig)].unit)}
drive: ${to_string(input.diskConfig[count.index % length(input.diskConfig)].drive)}
label: ${to_string(input.diskConfig[count.index % length(input.diskConfig)].label)}

We can also transform, where needed, values from integers to strings using the to_string() function.

Note: On hindsight, for unit we could of used a string input with an enum of values from 1-4 and this would have given the end user a drop down to select from like we did for the SCSI Controller.

As a reminder, for more detailed information on Expression Syntax, checkout VMware Aria Automation Assembler Expression Syntax on the VMware Documentation site.

Now, if we were to expand the code for both Cloud.vSphere.Machine and Cloud.vSphere.Disk just for vm[0], we would expect to see something like:

  vm[0]:
    type: Cloud.vSphere.Machine
    properties:
      image: windows
      flavor: small
      constraints:
        - tag: cloud:vmc
      attachedDisks:
        - source: ${resource.disk[0].id}
        - source: ${resource.disk[1].id}
 disk[0]:
    type: Cloud.vSphere.Disk
    properties:
        capacityGb: ${input.diskConfig[0].size}
        SCSIController: ${input.diskConfig[0].controller}
        unitNumber: ${input.diskConfig[0].unit}
        drive: ${input.diskConfig[0].drive}
        label: ${input.diskConfig[0].label}
 disk[1]:
    type: Cloud.vSphere.Disk
    properties:
        capacityGb: ${input.diskConfig[1].size}
        SCSIController: ${input.diskConfig[1].controller}
        unitNumber: ${input.diskConfig[1].unit}
        drive: ${input.diskConfig[1].drive}
        label: ${input.diskConfig[1].label}

Warning: It is important to understand that when using count to increment the number of Virtual Machines, you are effectively creating an array (or cluster) of Virtual Machine objects. When you add into this a dynamic, rather than static, allocation of disks, we are restricted to certain Day 2 actions for both the Deployment and Virtual Machine. For example, you cannot Update the Deployment (scale the number of Virtual Machines) or Unregister a Virtual Machine in the cluster.

Template Code

Adding that all in together and we get the following VMware Cloud Template YAML code:

Wrapping It All Up

As a recap, we have achieved great things in this series and we managed to achieve all of the requirements we set out. We have evolved our original VMware Cloud Template from deploying just a single Virtual Machine to being able to deploying multiple Virtual Machines with identical disk configuration which is chosen at request time. Along the way we looked at a number of different VMware Cloud Templates which solved different parts of the overall use case.

However, it is worth noting the warning above around the Cloud Template we have created. Whilst a dynamic Multi-VM, Multi-Disk cloud template may sound awesome, it does come with restrictions that may mean we need to find a better way.

Published on 29 March 2024 by Christopher Lewis. Words: 1820. Reading Time: 9 mins.

Blog Categories: VMware VMware Aria Automation Aria Automation