Getting domain name from a url using powershell?

Question

Basically I have a huge csv of phishing links and I'm trying to trim off https://www. and anything after .com .edu etc. so basically the ideal ouput of the powershell script would be a long list of urls all of which look something like google.com or microsoft.com so far I have imported the csv but everything I have tried either doesn't work or leaves the www on the beggining. Any help would be great. The csv im using is this: http://data.phishtank.com/data/online-valid.csv

$urls = Import-Csv -Path .\online-valid.csv | select -ExpandProperty "url"

run this [URI]'http://www.phishtank.com/phish_detail.php?phish_id=6429209' and you're half there. ;-) — Olaf
– Olaf, Commented Mar 3, 2020 at 2:44

Drew · Accepted Answer · 2020-03-03 03:51:41Z

1

The below will take your CSV and do magic for you. Have a play around with [Uri], it is very useful when parsing web links.

$csv = import-csv C:\temp\verified_online.csv

Foreach($Site in $csv) {
    $site | Add-Member -MemberType NoteProperty -Name "Host" -Value $(([Uri]$Site.url).Host -replace '^www\.')
}

$csv | Export-Csv C:\temp\verified_online2.csv -NoTypeInformation

Adjusted based on recommendation from Mklement0.

edited Mar 3, 2020 at 3:51

answered Mar 3, 2020 at 2:45

Drew

4,0682 gold badges13 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

mklement0 · Accepted Answer · 2020-03-03 17:32:21Z

1

A concise and fast alternative to Drew's helpful answer based on casting the URL strings directly to an array of [uri] (System.Uri) instances, and then trimming prefix www., if present, from their .Host (server name) property:

([uri[]] (Import-Csv .\online-valid.csv).url).Host -replace '^www\.'

Note that the -replace operator is regex-based, and regex ^www\. makes sure what www is only replaced at the start (^) of the string, and only if followed by a literal . (\.), in which case this prefix is removed (replaced with the implied empty string); if no such prefix is present, the input string is passed through as-is.

The solution reads the entire CSV file into memory at once, for convenience and speed, and outputs just the trimmed server names, as an array of strings.

edited Mar 3, 2020 at 17:32

answered Mar 3, 2020 at 3:41

mklement0

452k68 gold badges727 silver badges986 bronze badges

Collectives™ on Stack Overflow

Getting domain name from a url using powershell?

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related