Unable to fetch data using external data source

Question

I am trying to fetch values from azure using external data source in terraform. However, i dont understand what am i doing wrong when i try to export values using write-output, getting an error

data.external.powershell_test: data.external.powershell_test: command "Powershell.exe" produced invalid JSON: invalid character 'l' looking for beginning of object key string"

Below is my script

$vm=(Get-AzureRmVM -ResourceGroupName MFA-RG -Name vm2).name | convertTo-json
Write-Output "{""first"" : ""$vm""}"

Main.tf file

data "external" "powershell_test" {
  program = ["Powershell.exe", "./vm.ps1"]
}

output "value" {
  value = "${data.external.powershell_test.result.first}"
}

Can someone tel me what wrong with the script ? and if i am using write-out properly ?

Edited-------------

Below is the screenshot when i run vm.ps1 directly

Also, when i directly assign value to a variable as below, terraform is able to execute the code.

$vm = "testvm"
Write-Output "{""first"" : ""$vm""}"

What does the output look like if you run the vm.ps1 script directly? — ydaetskcoR
– ydaetskcoR, Commented Mar 13, 2019 at 16:00

Charles Xu · Accepted Answer · 2019-03-14 08:29:39Z

3

For your issue, you should change your PowerShell command like this:

$vm=(Get-AzureRmVM -ResourceGroupName MFA-RG -Name vm2).name | convertTo-json
Write-Output "{""first"" : $vm}"

And you could change the code in the data source like this or not, but I suggest you do this:

data "external" "powershell_test" {
      program = ["Powershell.exe", "${path.module}/vm.ps1"]
    }

The result on my side is below:

I use the new Azure PowerShell module Az and my code shows here:

PowerShell:

$vm=(Get-AzVM -ResourceGroupName charles -Name azureUbuntu18).name | convertTo-json
Write-Output "{""first"" : $vm}"

Terraform:

data "external" "powershell_test" {
  program = ["Powershell.exe", "${path.module}/vm.ps1"]
}

output "value" {
  value = "${data.external.powershell_test.result.first}"
}

edited Mar 14, 2019 at 8:29

answered Mar 14, 2019 at 6:41

Charles Xu

31.8k2 gold badges26 silver badges42 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

user2549572 Over a year ago

Thank you charles, your solution worked after installing Az module !! can you please tell me why same thing wasn't working on AzureRM module ?

Charles Xu Over a year ago

@user2549572 You also can do that through AzureRM module. The problem is that when you refer the VM in your Powershell, you'd better just change your code as I said in the answer.

user2549572 Over a year ago

But when i try using the code you shared of AzureRM module, it gives the same error but changing code to Az module works

Charles Xu Over a year ago

@user2549572 You can check if the AzureRM module gives you the thing as you expected.

user2549572 Over a year ago

When i run the vm.ps1 directly with AzureRM module it gives desired output ({"first" : "VM2"}) but when i run terraform apply , it gives the error (data.external.powershell_test: data.external.powershell_test: command "Powershell.exe" produced invalid JSON)

|

BMW · Accepted Answer · 2019-03-14 01:04:21Z

1

data.external.powershell_test.result is the only valid attribute, and it is map.

So the code will be changed to

output "value" {
  value = "${data.external.powershell_test.result['first']}"
}

Reference:

https://www.terraform.io/docs/configuration-0-11/interpolation.html#user-map-variables

answered Mar 14, 2019 at 1:04

BMW

45.6k13 gold badges105 silver badges123 bronze badges

1 Comment

ossentoo Over a year ago

this doesn't appear to be formatted correctly for me. I get this error: parse error at 1:38: expected expression but found invalid sequence "'"

redzack · Accepted Answer · 2021-02-09 18:53:51Z

Thanks Charles XU for the answer. I was looking for Azure Application Gateway and after lots of digging I ended up here, as Terraform is yet to provide a data source for Azure Application Gateway. However, the same can be done using shell script and azure rest API.

Using Shell Script

appgw.sh

#!/bin/bash

#Linux: Requires Azure cli and jq to be available

#Getting Application Gateway ID using azure application gateway rest API, az cli as data source doesn't exist for it.
appgwid=$(az rest -m get --header "Accept=application/json" -u 'https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/newrg/providers/Microsoft.Network/applicationGateways?api-version=2020-07-01' | jq '.value[].id')

#Terraform External Data Source requires an output, else it will return in unmarshal json error with }
echo "{\"appgw_id\" : $appgwid}"

data.tf

data "external" "appgw_id_sh" {
      program = ["/bin/bash", "${path.module}/appgw.sh"]
}

outputs.tf

output "appgw_id_sh" {
  value = data.external.appgw_id_sh.result.appgw_id
}

Using Powershell

appgw.ps1

#Windows: Require Azure Powershell to be available
#1. Install-Module -Name PowerShellGet -Force
#2. Install-Module -Name Az -AllowClobber

#Getting Application Gateway ID using AzApplicationGateway AzResource as data source doesn't exist for it.
$appGw = (Get-AzApplicationGateway -Name "appgw-name" -ResourceGroupName "appgw-rg-name").id | convertTo-json

#Terraform External Data Source requires an output, else it will return in unmarshal json error with }
Write-Output "{""appgw_id"" : $appgw}"

data.tf

data "external" "appgw_id_ps" {
      program = ["Powershell.exe", "${path.module}/appgw.ps1"]
}

outputs.tf

output "appgw_id_ps" {
  value = data.external.appgw_id_ps.result.appgw_id
}

Mike T · Accepted Answer · 2022-02-20 16:30:18Z

0

FYI - when I use powershell 7, I need to use pwsh.exe instead of powershell.exe inside program code.

data "external" "powershell_test" {
  program = ["**pwsh.exe**", "${path.module}/vm.ps1"]
}

answered Feb 20, 2022 at 16:30

Mike T

1831 gold badge2 silver badges13 bronze badges

Collectives™ on Stack Overflow

Unable to fetch data using external data source

4 Answers 4

8 Comments

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

8 Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related