PowerShell Snippet: Reusing Functions in multi-threaded scripts

I had a use case where I wanted to deploy and configure multiple virtual machines in parallel. Each virtual machine required the execution of a number of existing functions to complete the task, and a series of variables to be passed to each function to accomplish that.

There is not a lot of good information out there on how to reuse those functions. Many online threads will advocate that you write (or duplicate) the function inside the script block you intend to use when executing the thread you are about to spin out, but all of that just leads to duplicate and less reusable code.

Let’s build up a simple example, so you can see what I mean.

The Problem

Let’s assume that I have written a script that has a series of functions designed to help me deploy the virtual machines. Some virtual machines are basic, and others require a further tweak to add an additional hard drive. So I have two different use cases, and I want to be as efficient with my code as possible and maximise my code reuse

#Region Define Functions

Function New-vCenterConnection
{
    Param (
    [Parameter (Mandatory = $true)] [String]$vCenterFqdn,
    [Parameter (Mandatory = $true)] [String]$vCenterPassword,
    [Parameter (Mandatory = $true)] [String]$vCenterUser
    )
    Connect-VIServer -Server $vCenterFqdn -user $vCenterUser -pass $vCenterPassword
}

Function New-VirtualMachine
{
    Param (
    [Parameter (Mandatory = $true)] [String]$virtualMachineName
    )
    New-VM -VMhost (Get-VMHost | Get-Random) -Name $virtualMachineName -Datastore (Get-Datastore)[0].name -DiskGB 100 -DiskStorageFormat Thin -MemoryGB 8 -NumCpu 4 -portgroup "VM Network" -GuestID "vmkernel7Guest"-Confirm:$false
}

Function Set-SpecialVMConfiguration
{
    Param (
    [Parameter (Mandatory = $true)] [String]$virtualMachineName
    )
    Get-VM $virtualMachineName | New-HardDisk -CapacityGB "10" -StorageFormat Thin -Confirm:$false
}

#EndRegion Define Functions

#Region Execute Script

#Define VM name arrays
$basicVMs =@("BasicVM-001","BasicVM-002","BasicVM-003","BasicVM-004","BasicVM-005")
$advancedVMs =@("AdvancedVM-001","AdvancedVM-002","AdvancedVM-003","AdvancedVM-004","AdvancedVM-005")

#Execute functions to create both sets of VMs
New-vCenterConnection -vCenterFqdn "myvcenter.mydomain.com" -vCenterUser "Administrator@vsphere.local" -vCenterPassword "mycomplexpassword"
Foreach ($basicVM in $basicVMs)
{
    New-VirtualMachine -virtualMachineName $basicVM
}
Foreach ($advancedVM in $advancedVMs)
{
    New-VirtualMachine -virtualMachineName $advancedVM
    Set-SpecialVMConfiguration -virtualMachineName $advancedVM
}

#Region Execute Script

The above code will create 10 virtual machines, 5 basic, and 5 advanced, but it will do it all in a serialised fashion, so all 5 basic VMs will be created before we even start the 5 advanced VMs. What if we wanted to start each VM and do a monitored unattended installation of each which takes 30 mins? Then my script is going to be running for 5 hours. Not ideal. It would be great if we could speed it up by deploying each of the VMs in parallel.

Let’s take our existing script and add a script block for each type of deployment, and use Start-Job. You might think that something like this would work:

#Region Define Functions

Function New-vCenterConnection
{
    Param (
    [Parameter (Mandatory = $true)] [String]$vCenterFqdn,
    [Parameter (Mandatory = $true)] [String]$vCenterPassword,
    [Parameter (Mandatory = $true)] [String]$vCenterUser
    )
    Connect-VIServer -Server $vCenterFqdn -user $vCenterUser -pass $vCenterPassword
}

Function New-VirtualMachine
{
    Param (
    [Parameter (Mandatory = $true)] [String]$virtualMachineName
    )
    New-VM -VMhost (Get-VMHost | Get-Random) -Name $virtualMachineName -Datastore (Get-Datastore)[0].name -DiskGB 100 -DiskStorageFormat Thin -MemoryGB 8 -NumCpu 4 -portgroup "VM Network" -GuestID "vmkernel7Guest"-Confirm:$false
}

Function Set-SpecialVMConfiguration
{
    Param (
    [Parameter (Mandatory = $true)] [String]$virtualMachineName
    )
    Get-VM $virtualMachineName | New-HardDisk -CapacityGB "10" -StorageFormat Thin -Confirm:$false
}

Function New-MonitoredOSInstallation
{
Param (
    [Parameter (Mandatory = $true)] [String]$virtualMachineName
    )
    Start-VM $virtualMachineName
    # Plus a series of really complex operations to setup and monitor the machine Guest OS Installation
}

#EndRegion Define Functions

#Region Execute Script

#Define Script Blocks
$basicVMScriptBlock = {
    New-VirtualMachine -virtualMachineName $args[0]
    New-MonitoredOSInstallation -virtualMachineName $args[0]
 }
$advancedVMScriptBlock = {
    New-VirtualMachine -virtualMachineName $args[0]
    Set-SpecialVMConfiguration -virtualMachineName $args[0]
    New-MonitoredOSInstallation -virtualMachineName $args[0]
 }

#Define VM name arrays
$basicVMs =@("BasicVM-001","BasicVM-002","BasicVM-003","BasicVM-004","BasicVM-005")
$advancedVMs =@("AdvancedVM-001","AdvancedVM-002","AdvancedVM-003","AdvancedVM-004","AdvancedVM-005")

#Execute functions to create both sets of VMs
New-vCenterConnection -vCenterFqdn "myvcenter.mydomain.com" -vCenterUser "Administrator@vsphere.local" -vCenterPassword "mycomplexpassword"
Foreach ($basicVM in $basicVMs)
{
    Start-Job -scriptblock $basicVMScriptBlock -argumentlist ($basicVM)
}
Foreach ($advancedVM in $advancedVMs)
{
    Start-Job -scriptblock $advancedVMScriptBlock -argumentlist ($advancedVM)
}
Get-Job | Receive-Job -Wait -Auto RemoveJob

#EndRegion Execute Script

The problem with the above approach is that none of the function definitions we have spent so much time crafting are available inside the new thread, so even though the scriptblocks have the correct sequence of steps, you are going to get a bunch of ‘cmdlet not found’ type responses when you try to run your script. Also, if you notice in the above, in an effort to only connect to vCenter once, we left the New-vCenterConnection function outside the scriptblock, so even though we have a vCenter connection in parent PowerShell session, we would have no connection in the child jobs.

As I mentioned earlier, the approach many online threads will advocate is to put the function definition inside the scriptblock, but we have two scriptblocks! That would mean duplicating the following functions in both blocks:

New-vCenterConnection
New-VirtualMachine
New-MonitoredOSInstallation

Look how messy and redundant our scriptblocks look now….and the problem only gets worse as you add functions or different sequences of functions.

#Region Execute Script

#Define Script Blocks
$basicVMScriptBlock = {
    #Define Functions within scriptblock
    Function New-vCenterConnection
    {
        Param (
        [Parameter (Mandatory = $true)] [String]$vCenterFqdn,
        [Parameter (Mandatory = $true)] [String]$vCenterPassword,
        [Parameter (Mandatory = $true)] [String]$vCenterUser
        )
        Connect-VIServer -Server $vCenterFqdn -user $vCenterUser -pass $vCenterPassword
    }

    Function New-VirtualMachine
    {
        Param (
        [Parameter (Mandatory = $true)] [String]$virtualMachineName
        )
        New-VM -VMhost (Get-VMHost | Get-Random) -Name $virtualMachineName -Datastore (Get-Datastore)[0].name -DiskGB 100 -DiskStorageFormat Thin -MemoryGB 8 -NumCpu 4 -portgroup "VM Network" -GuestID "vmkernel7Guest"-Confirm:$false
    }

    Function New-MonitoredOSInstallation
    {
        Param (
        [Parameter (Mandatory = $true)] [String]$virtualMachineName
        )
        Start-VM $virtualMachineName
        # Plus a series of really complex operations to setup and monitor the machine Guest OS Installation
    }

    #Call Functions within scriptblock
    New-vCenterConnection -vCenterFqdn $args[1] -vCenterPassword $args[2] -vCenterUser $args[3]
    New-VirtualMachine -virtualMachineName $args[0]
    New-MonitoredOSInstallation -virtualMachineName $args[0]
}

$advancedVMScriptBlock = {
    #Define Functions within scriptblock
    Function New-vCenterConnection
    {
        Param (
        [Parameter (Mandatory = $true)] [String]$vCenterFqdn,
        [Parameter (Mandatory = $true)] [String]$vCenterPassword,
        [Parameter (Mandatory = $true)] [String]$vCenterUser
        )
         Connect-VIServer -Server $vCenterFqdn -user $vCenterUser -pass $vCenterPassword
    }
    
    Function New-VirtualMachine
    {
        Param (
        [Parameter (Mandatory = $true)] [String]$virtualMachineName
        )
        New-VM -VMhost (Get-VMHost | Get-Random) -Name $virtualMachineName -Datastore (Get-Datastore)[0].name -DiskGB 100 -DiskStorageFormat Thin -MemoryGB 8 -NumCpu 4 -portgroup "VM Network" -GuestID "vmkernel7Guest"-Confirm:$false
    }
    
    Function New-MonitoredOSInstallation
    {
        Param (
        [Parameter (Mandatory = $true)] [String]$virtualMachineName
        )
        Start-VM $virtualMachineName
        # Plus a series of really complex operations to setup and monitor the machine Guest OS Installation
    }

    Function Set-SpecialVMConfiguration
    {
        Param (
        [Parameter (Mandatory = $true)] [String]$virtualMachineName
        )
        Get-VM $virtualMachineName | New-HardDisk -CapacityGB "10" -StorageFormat Thin -Confirm:$false
    }

    #Call Functions within scriptblock
    New-vCenterConnection -vCenterFqdn $args[1] -vCenterPassword $args[2] -vCenterUser $args[3]
    New-VirtualMachine -virtualMachineName $args[0]
    Set-SpecialVMConfiguration -virtualMachineName $args[0]
    New-MonitoredOSInstallation -virtualMachineName $args[0]
}

#Define VM name arrays
$basicVMs =@("BasicVM-001","BasicVM-002","BasicVM-003","BasicVM-004","BasicVM-005")
$advancedVMs =@("AdvancedVM-001","AdvancedVM-002","AdvancedVM-003","AdvancedVM-004","AdvancedVM-005")

#Execute functions to create both sets of VMs
$vCenterFqdn = "myvcenter.mydomain.com"
$vCenterUser = "Administrator@vsphere.local"
$vCenterPassword = "mycomplexpassword"

Foreach ($basicVM in $basicVMs)
{
    Start-Job -scriptblock $basicVMScriptBlock -argumentlist ($basicVM,$vCenterFqdn,$vCenterUser,$vCenterPassword)
}
Foreach ($advancedVM in $advancedVMs)
{
    Start-Job -scriptblock $advancedVMScriptBlock -argumentlist ($advancedVM,$vCenterFqdn,$vCenterUser,$vCenterPassword)
}
Get-Job | Receive-Job -Wait -Auto RemoveJob

#Region Execute Script

This time, we duplicated the functions, and we moved the New-vCenterConnection inside the script block to ensure we make a connection to vCenter in each thread, thereby necessitating that we pass more parameters when running Start-Job. While the above code will work, its not that efficient, and it you need to make a change to one of the functions, you would need to remember to do it everywhere you had to duplicate the function in the code block.

The Solution

The solution is strikingly simple. So much so I’m surprised it’s not offered up everywhere the question has been asked.

Use a PowerShell Module

In short, you place your functions in a module and import the module in your scriptblock. Now your functions are available and there is no duplication. If you’re not sure how to create a PowerShell module check out this link.

Let’s see what that looks like in code

Contents of MyModule.psm1 file

#Region Define Functions

Function New-vCenterConnection
{
    Param (
    [Parameter (Mandatory = $true)] [String]$vCenterFqdn,
    [Parameter (Mandatory = $true)] [String]$vCenterPassword,
    [Parameter (Mandatory = $true)] [String]$vCenterUser
    )
    Connect-VIServer -Server $vCenterFqdn -user $vCenterUser -pass $vCenterPassword
}

Function New-VirtualMachine
{
    Param (
    [Parameter (Mandatory = $true)] [String]$virtualMachineName
    )
    New-VM -VMhost (Get-VMHost | Get-Random) -Name $virtualMachineName -Datastore (Get-Datastore)[0].name -DiskGB 100 -DiskStorageFormat Thin -MemoryGB 8 -NumCpu 4 -portgroup "VM Network" -GuestID "vmkernel7Guest"-Confirm:$false
}

Function New-MonitoredOSInstallation
{
    Param (
    [Parameter (Mandatory = $true)] [String]$virtualMachineName
    )
    Start-VM $virtualMachineName
    # Plus a series of really complex operations to setup and monitor the machine Guest OS Installation
}

Function Set-SpecialVMConfiguration
{
    Param (
    [Parameter (Mandatory = $true)] [String]$virtualMachineName
    )
    Get-VM $virtualMachineName | New-HardDisk -CapacityGB "10" -StorageFormat Thin -Confirm:$false
}

#EndRegion Define Functions

Contents of MyScript.ps1 file

#Region Execute Script

#Define Script Blocks
$basicVMScriptBlock = {
    Import-Module $PSScriptRoot\MyModule.psm1
    New-vCenterConnection -vCenterFqdn $using:vCenterFqdn -vCenterUser $using:vCenterUser -vCenterPassword $using:vCenterPassword
    New-VirtualMachine -virtualMachineName $using:basicVM
    New-MonitoredOSInstallation -virtualMachineName $using:basicVM
 }
$advancedVMScriptBlock = {
    Import-Module $PSScriptRoot\MyModule.psm1
    New-vCenterConnection -vCenterFqdn $using:vCenterFqdn -vCenterUser $using:vCenterUser -vCenterPassword $using:vCenterPassword
    New-VirtualMachine -virtualMachineName $using:advancedVM
    Set-SpecialVMConfiguration -virtualMachineName $using:advancedVM
    New-MonitoredOSInstallation -virtualMachineName $using:advancedVM
 }

#Define VM name arrays
$basicVMs =@("BasicVM-001","BasicVM-002","BasicVM-003","BasicVM-004","BasicVM-005")
$advancedVMs =@("AdvancedVM-001","AdvancedVM-002","AdvancedVM-003","AdvancedVM-004","AdvancedVM-005")

#Execute functions to create both sets of VMs
$vCenterFqdn = "myvcenter.mydomain.com"
$vCenterUser = "Administrator@vsphere.local"
$vCenterPassword = "mycomplexpassword"
Foreach ($basicVM in $basicVMs)
{
    Start-Job -scriptblock $basicVMScriptBlock -argumentlist ($basicVM,$vCenterFqdn,$vCenterUser,$vCenterPassword)
}
Foreach ($advancedVM in $advancedVMs)
{
    Start-Job -scriptblock $advancedVMScriptBlock -argumentlist ($advancedVM,$vCenterFqdn,$vCenterUser,$vCenterPassword)
}
Get-Job | Receive-Job -Wait -Auto RemoveJob

#EndRegion Execute Script

The simple ‘Import-Module’ statement in each block gets us around all the duplication, and makes every function you write in your module available in each thread you spin out.

One other thing I’ve done in the ps1 file this time is to switch from using the $args[0] format to the $using format in the script blocks to reference the variables passed into it with the Start-Job command. IMHO this method is a far easier way of matching what was was passed into Start-Job to the parameters that need to be passed to each function within the new threads.

Happy threading!

The Problem

The Solution

Contents of MyModule.psm1 file

Contents of MyScript.ps1 file

Share this:

Related

Leave a comment Cancel reply