Can't execute AWS CLI command with user_data in Terraform

Question

I am using Terraform to spin up an EC2-instance where I have defined some user_data. In the user_data I download AWS CLI and set up the config file and credentials file. I then want to grab a file from my s3 bucket. This is what I've written so far:

data "template_file" "scripts" {
  template = <<EOF
<powershell>
$dlurl = "installpathtoawscli.msi"
$installerPath = Join-Path $env:TEMP (Split-Path $dlurl -Leaf)
Invoke-WebRequest $dlurl -OutFile $installerPath
Start-Process -FilePath msiexec -Args "/i $installerPath /passive" -Verb RunAs -Wait
mkdir C:\temp
New-Item -Name 'b.bat' -Path 'C:\temp' -Value 'aws s3 cp s3://mybucketname/myfile.extension C:\temp\extension > C:\temp\test.txt'
New-Item -Name 'a.bat' -Path 'C:\Users\Administrator' -Value 'mkdir C:\Users\Administrator\.aws
echo [default]>> C:\Users\Administrator\.aws\credentials
echo aws_access_key_id = MY_ACCESS_KEY>> C:\Users\Administrator\.aws\credentials
echo aws_secret_access_key = MY_SECRET_KEY>> C:\Users\Administrator\.aws\credentials
echo [default]>> C:\Users\Administrator\.aws\config
echo region = MY_REGION>> C:\Users\Administrator\.aws\config
echo output = MY_OUTPUT>> C:\Users\Administrator\.aws\config'
C:\Users\Administrator\a.bat
C:\temp\b.bat
</powershell>
EOF
}

b.bat is executed and a txt file is created, however, it is empty and no file is grabbed from the s3 bucket. If I try to execute it manually (by RDPing to the instance) it grabs the file. Any ideas why this is happening?

When you RDP, are you just executing b.bat as is and under the same login as the terraform? If that's not an option, perhaps redirect the output of b.bat to a file and see if there are any errors etc. — T.H.
– T.H., Commented Sep 18, 2020 at 16:01
Yes, same login. I return a txt file with the output of the command but it is empty every time I run it from Terraform. When I do it manually, the txt file will not be empty — Chronospear
– Chronospear, Commented Sep 19, 2020 at 6:30
When you say manually, do you mean executing b.bat as is, or just the command itself? If b.bat as is works, then also perhaps try it from within powershell. If that also works then capturing the output of b.bat (not just the output of the aws command within it) during the initial run might give a clue to what's happening. — T.H.
– T.H., Commented Sep 19, 2020 at 12:28
It works either way, both when running it from the command itself and executing it manually. Furthermore, I've tried to run it without the .bat file without any success e.g. aws s3 cp s3://mybucketname/myfile.extension C:\temp\extension > C:\temp\test.txt | cmd.exe from <powershell> and also by aws s3 cp s3://mybucketname/myfile.extension C:\temp\extension > C:\temp\test.txt from <script>. — Chronospear
– Chronospear, Commented Sep 20, 2020 at 9:25
Perhaps try adding a long pause before b.bat exectures in case it's some time-dependent issue? Also, I hadn't considered the following when suggesting capturing the output: support.microsoft.com/en-gb/help/110930/…. — T.H.
– T.H., Commented Sep 20, 2020 at 20:07

Chronospear · Accepted Answer · 2020-09-24 08:00:22Z

2

I found the solution after several days of trying.

The AWS.exe location was not being registered in the PATH environment yet. Restarting the command prompt would do the trick, however, Terraform can't do this, thus explaining why it worked when I did it manually.

I had to provide the full path for AWS.exe in order to make it work e.g. C:\Program Files\Amazon\AWSCLI\bin\aws.exe or C:\Program Files\Amazon\AWSCLIV2\aws.exe depending on the version of AWS CLI.

answered Sep 24, 2020 at 8:00

Chronospear

538 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Can't execute AWS CLI command with user_data in Terraform

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related