122

I have a URL to a CSV file which, in a browser, I can download and open without issue.

I'm trying to download this file using PowerShell without success. I tried using Invoke-WebRequest, Start-BitsTransfer and using a webrequest object but no luck there.

Improve this question
1

4 Answers 4

Reset to default
187

Invoke-WebRequest comes with a parameter to store its result in a file: -OutFile

Invoke-WebRequest $myDownloadUrl -OutFile c:\file.ext

If you need authorization before you can send a request like this:

Invoke-WebRequest $myAuthUrl /* whatever is neccesary to login */ -SessionVariable MySession
Invoke-WebRequest $myDownloadUrl -WebSession $MySession

To determine the layout of the form where the login happens, you can use Invoke-WebRequests return object. It'll collect information about forms and fields on the HTML (might be Windows only). Mileage of logging in may vary with things like Two-Factor-Auth active or not. Probably you can create some secret link to your file which does not need Auth or possibly google allows you to create a private access token of some sort, which can be send aus Authorization-Header alongside your request.

Improve this answer
Sign up to request clarification or add additional context in comments.

10 Comments

Works fine here .. CSV with No headers and one row with three test-entries
@JustAGuy: It works with (at least) Windows PowerShell v5.1 / PowerShell Core v6.1.0-preview.3. If it doesn't work for you, please add details about your environment to your question.
@JustAGuy This does work here also, you may not have rights to write to C:\file.ext try Invoke-WebRequest 'https://docs.google.com/spreadsheets/d/1soyIpcac2hOb-5AdImm1LNCFs9jFiiyMhuqCayfpxSw/export?format=csv' -OutFile '$($Env:temp)\test.csv' to store the file in the temp folder.
I have major 5 minor 1 installed. How can I tell if it's 5.1? On my station it ends up as core html and not the actual file.
Have you quoted the URL? Perhaps there is a problem? $PSVersionTable tells you the Versions
|
47

TLDR answers*:

Method 1 (good for most cases), by default synchronous**

Invoke-WebRequest $url -OutFile $path_to_file

If you get error "...Could not create SSL/TLS secure channel" see Powershell Invoke-WebRequest Fails with SSL/TLS Secure Channel

Method 2 (for JSON/XML data), by default synchronous**

Invoke-RestMethod $url -OutFile $path_to_file

This is a wrapper around Invoke-WebRequest, with some additional formatting for JSON results. (See documentation here and here.)

Method 3, by default synchronous**

(New-Object System.Net.WebClient).DownloadFile($url, $path_to_file)

Method 4, asynchronous for when you don't mind waiting

Import-Module BitsTransfer
Start-BitsTransfer -Source $url -Destination $path_to_file

It uses the BITS service to download data when other processes don't need bandwidth (but as a result may be slower).

Notes:

*: This answer is for those that google for "how to download a file with PowerShell".

**: Read the help pages if you want asynchronous downloading

Improve this answer

3 Comments

I would guess that you were downvoted for not 'solving' the problem the user had. They indicated they had already tried the three methods you've suggested. Sounds like they were getting a login.html instead of a file.csv. They didn't specify that issue in their original question which they should have. However, it also seems like you didn't carefully read their question details and the other answer conversation before answering an already answered question. I've upvoted as your answer is helpful in similar contexts (but not this user's specific one).
Thanks for the thorough response duct_tape_coder! Your time is greatly appreciated. Indeed my answer was more towards the Google reader that searched for "how to download with powershell" than to the original author. There's a conflict here and I can understand the POV for downvoting. I also noticed that it is already answered but I love terse (TLDR) answers so I often add them to help others with similar taste.
Thank you for the answer, I used Method 3 because Core server throw back that Internet Explorer isn't available when using invoke-* variations
5 
$url = "https://www.somedomain.com/file.bin"
$dest = "C:\temp\testfiles.bin"
Invoke-RestMethod -Uri $url -OutFile $dest
Improve this answer

1 Comment

Thank you for contributing to the Stack Overflow community. This may be a correct answer, but it’d be really useful to provide additional explanation of your code so developers can understand your reasoning. This is especially useful for new developers who aren’t as familiar with the syntax or struggling to understand the concepts. Would you kindly edit your answer to include additional details for the benefit of the community?
0

For a while now I've been using a PS script to download PowerBI bi-monthly and using the BITS, it's been pretty solid and now so much stronger now since I removed the -Asynchronous at the end of the Start-BitsTransfer

$url = "https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup.exe"
$output = "%RandomPath%\PowerBI Pro\PBIDesktopSetup.exe"
$start_time = Get-Date

Import-Module BitsTransfer
Start-BitsTransfer -Source $url -Destination $output

#Commented out below because it kept creating "Tmp files"
#Start-BitsTransfer -Source $url -Destination $output -Asynchronous
Improve this answer

1 Comment

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.