Bash script execute wget with a variable inside it

Question

I'm trying to execute a wget command with a variable inside it but it just ignores it, any idea what am I doing wrong?

#!/bin/bash

URL=http:://www.myurl.com

echo $(date) 'Running wget...'
wget -O - -q "$URL/something/something2"

What makes you think it is being ignored? What is happening exactly? — Etan Reisner
– Etan Reisner, Commented May 7, 2015 at 16:53
it just keeps running, the command im running is supposed to create a log file and it doesnt...maybe I should remove the -O -q flags? — Broshi
– Broshi, Commented May 7, 2015 at 16:54
-O - will spit the retrieved document to standard output so you should see output if it is getting the page. You could remove -q to see if wget says anything about what is going on. But this sounds like it might just be taking wget a while to connect. How long have you waited? — Etan Reisner
– Etan Reisner, Commented May 7, 2015 at 16:57
aha, now I see the error: Resolving http (http)... failed: Name or service not known. wget: unable to resolve host address http and it added ftp:// prefix to the real http:// address... — Broshi
– Broshi, Commented May 7, 2015 at 16:58

Dave McMordie · Accepted Answer · 2015-05-07 17:03:00Z

8

Four things:

Add quotes around your URL: http:://www.myurl.com ==> "http:://www.myurl.com"
Remove the double colon: "http:://www.myurl.com" ==> "http://www.myurl.com"
Get rid of the extra flags and hyphen on the wget command: "wget -O - -q "$URL/something/something2"" ==> wget "$URL/something/something2"
Add curly braces around your variable: "wget "$URL/something/something2"" ==> "wget "${URL}/something/something2""

This works:

#!/bin/bash

URL="http://www.google.com"

echo $(date) 'Running wget...'
wget "${URL}"

answered May 7, 2015 at 17:03

Dave McMordie

5734 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Kobap Bopy · Accepted Answer · 2021-02-05 13:31:26Z

1

I use that code in IPython (colab):


URL = 'http:://www.myurl.com'
!wget {URL}

I wrote this answer because was searching it!)

answered Feb 5, 2021 at 13:31

Kobap Bopy

918 bronze badges

Comments

David C. Rankin · Accepted Answer · 2015-05-07 17:40:51Z

Another handy option in Bash (or other shells) is to create a simple helper function that calls wget with the common options required by nearly all sites and any specific options you generally use. This reduces the typing involved and can also be useful in your scripts. I place the following in my ~/.bashrc to make it available to all shells/subshells. It validates input, checks that wget is available, and then passes all command line arguments to wget with the default options set in the script:

wgnc () {
    if [ -z $1 ]; then
        printf "  usage:  wg <filename>\t\t(runs wget --no-check-certificate --progress=bar)\n"
    elif ! type wget &>/dev/null; then
        printf "  error: 'wget' not found on system\n"
    else
        printf "  wget --no-check-certificate --progress=bar %s\n" "$@"
        wget --no-check-certificate --progress=bar "$@"
    fi
}

You can cut down typing even more by aliasing the function further. I use:

alias wg='wgnc'

Which reduces the normal wget --no-check-certificate --progress=bar URL to simply wg URL. Obviously, you can set the options to suit your needs, but this is a further way to utilize wget in your scripts.

ATES · Accepted Answer · 2024-05-11 04:20:48Z

0

for Kaggle notebook:

url = "https://google.com"
name = "file.f"
!wget "$url" -O "$name"

answered May 11, 2024 at 4:20

ATES

3372 gold badges15 silver badges34 bronze badges

Collectives™ on Stack Overflow

Bash script execute wget with a variable inside it

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related