TFS Build Solution failure: UnauthorizedAccess Microsoft.PowerShell.Commands.ImportModuleCommand

I'm attempting to set up our first build agent for TFS 2017 On-Premise. I've taken a 2012 R2 server, installed the agent and Visual Studio 2017. The agent appears in TFS with the correct capabilities. I've taken one of the default templates for building ASP.net projects and made the minimum changes possible to it. On the first run, I can see the first few steps of getting sources and updating NuGet packages complete successfully. However the build solution step fails, and I'm struggling to understand what the issue may be.

The build log shows the following:

2017-11-15T13:16:10.3583612Z ##[section]Starting: Build solution
2017-11-15T13:16:10.3583612Z ==============================================================================
2017-11-15T13:16:10.3583612Z Task         : Visual Studio Build
2017-11-15T13:16:10.3583612Z Description  : Build with MSBuild and set the Visual Studio version property
2017-11-15T13:16:10.3583612Z Version      : 1.119.0
2017-11-15T13:16:10.3583612Z Author       : Microsoft Corporation
2017-11-15T13:16:10.3583612Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613727)
2017-11-15T13:16:10.3583612Z ==============================================================================
2017-11-15T13:16:10.8588602Z Import-Module : AuthorizationManager check failed.
2017-11-15T13:16:10.8588602Z At line:1 char:453
2017-11-15T13:16:10.8588602Z + . ([scriptblock]::Create('if (!$PSHOME) { $null = Get-Item -LiteralPath ''variab ...
2017-11-15T13:16:10.8588602Z + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2017-11-15T13:16:10.8588602Z     + CategoryInfo          : SecurityError: (:) [Import-Module], PSSecurityException
2017-11-15T13:16:10.8588602Z     + FullyQualifiedErrorId : UnauthorizedAccess,Microsoft.PowerShell.Commands.ImportModuleCommand
2017-11-15T13:16:10.9677474Z Invoke-VstsTaskScript : The term 'Invoke-VstsTaskScript' is not recognized as the name of a cmdlet, function, script 
2017-11-15T13:16:10.9677474Z file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct 
2017-11-15T13:16:10.9677474Z and try again.
2017-11-15T13:16:10.9677474Z At line:1 char:786
2017-11-15T13:16:10.9677474Z + ... tlyContinue' ; Invoke-VstsTaskScript -ScriptBlock ([scriptblock]::Create('. ''\\ ...
2017-11-15T13:16:10.9677474Z +                    ~~~~~~~~~~~~~~~~~~~~~
2017-11-15T13:16:10.9677474Z     + CategoryInfo          : ObjectNotFound: (Invoke-VstsTaskScript:String) [], CommandNotFoundException
2017-11-15T13:16:10.9677474Z     + FullyQualifiedErrorId : CommandNotFoundException
2017-11-15T13:16:10.9677474Z  
2017-11-15T13:16:10.9989971Z ##[error]Exit code 1 returned from process: file name 'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe', arguments '-NoLogo -Sta -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". ([scriptblock]::Create('if (!$PSHOME) { $null = Get-Item -LiteralPath ''variable:PSHOME'' } else { Import-Module -Name ([System.IO.Path]::Combine($PSHOME, ''Modules\Microsoft.PowerShell.Management\Microsoft.PowerShell.Management.psd1'')) ; Import-Module -Name ([System.IO.Path]::Combine($PSHOME, ''Modules\Microsoft.PowerShell.Utility\Microsoft.PowerShell.Utility.psd1'')) }')) 2>&1 | ForEach-Object { Write-Verbose $_.Exception.Message -Verbose } ; Import-Module -Name '\\hidden-unc-path\_tasks\VSBuild_71a9a2d3-a98a-4caa-96ab-affca411ecda\1.119.0\ps_modules\VstsTaskSdk\VstsTaskSdk.psd1' -ArgumentList @{ NonInteractive = $true } -ErrorAction Stop ; $VerbosePreference = 'SilentlyContinue' ; $DebugPreference = 'SilentlyContinue' ; Invoke-VstsTaskScript -ScriptBlock ([scriptblock]::Create('. ''\\hidden-unc-path\_tasks\VSBuild_71a9a2d3-a98a-4caa-96ab-affca411ecda\1.119.0\VSBuild.ps1'''))"'.
2017-11-15T13:16:10.9989971Z ##[section]Finishing: Build solution