Get a file from a URL with PowerShell

Question

I am receiving an email that contains a link to a website that immediately starts a download of a file. I am able to successfully get the email and the URL, and when I paste the URL into the browser it automatically starts a download. The web page is below:

Unfortunately, the file can only be sent in an .xls format, but my end goal is to convert it to a CSV.

I know that Invoke-WebRequest is supposed to do this, and my command for that is:

Invoke-WebRequest -Uri $ExcelLink -OutFile 'C:\Temp\FileName.xls'.

I have also tried the following:

(New-Object System.Net.WebClient).DownloadFile($ExcelLink,'C:\Temp\FileName.xls')

I have tried setting the export to be both .xls and .csv, and it appears I can only get the raw HTML code, instead of the file to download. In the screenshots below the left is exporting as .csv, and the right is .xls:

I have done a decent amount of research already, and the most helpful link was this Stackoverflow post.

The link from the email does not contain the file name. I stripped out a good amount from the URL, but it looks something like this:

https://_______.com/f/a/vl6K...hRdg~~/AA...gA~/RgRnjCy...QAAAM-

I have tried adding the file name to the end of the URL and for some reason it just redirects to Google.

Does anybody know of a way to get only the file content that automatically starts downloading when entering the URL in a browser?

It sounds like the link in your email isn't a link to a binary file - it's a link to a webpage (the page in your first screenshot that says "Download file...") that contains another link (the one in the "click here" text) which is the link to the actual binary file. What you'll need to do is download the source html of the webpage with Invoke-WebRequest using the link in your email and process that to extract the download link, and then you can make a second call to Invoke-WebRequest with the actual binary file link... — mclayton
– mclayton, Commented Jan 19, 2024 at 17:00
I looked into that. If I save Invoke-WebRequest into a variable called $Request, the value of $Request.Links.outerHTML is: <a href="#!" onclick="fileDelivery.downloadFile();">click here</a>. I tried appending #! to the end of the URL and got the same results — Andrew Draper
– Andrew Draper, Commented Jan 19, 2024 at 17:08
From the looks of things, your https://_______.com/... link in the email is the webpage - you need to download that first and to look deeper into how it triggers the automatic binary file download - something there knows what the link is for the target file. If the page author is trying to obfuscate it you might find it's hidden away in a http header, or another external javascript file or something else in the page source, and extracting it could require you to emulate a subset of the browser's functionality, which may end up being non-trivial... — mclayton
– mclayton, Commented Jan 19, 2024 at 18:18
Thank you @mclayton. It turns out the RawContent attribute contained some javascript that contained the URL. Thanks for pointing me in the right direction. — Andrew Draper
– Andrew Draper, Commented Jan 19, 2024 at 19:22

Andrew Draper · Accepted Answer · 2024-01-19 19:21:59Z

For anybody finding this in the future, I was able to find a solution to this. First, do an Invoke-WebRequest on the URL from the email. Then, look into the RawContent attribute. This may be different for each request, but for my specific request there was section of javascript where there was a variable defined as downloadUrl. Using that URL in another Invoke-WebRequest I was able to successfully download the file.

Here is some sample code, which works for my specific website request. Hopefully this will help somebody troubleshoot in the future.

# $ExcelLink is the URL included in the email, which opens the web page and prompts the automatic download
$Request = Invoke-WebRequest -Uri $ExcelLink
# The parentheses will grab the URL string variable in the first regex group
$DownloadUrlRegex = "var downloadUrl = '(\S+)';"
$Request.RawContent -match $DownloadUrlRegex | Out-Null
$DownloadUrl = $Matches[1]
Invoke-WebRequest -Uri $DownloadURL -OutFile $Destination

Collectives™ on Stack Overflow

Get a file from a URL with PowerShell

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related