Python 3 - urllib, HTTP Error 407: Proxy Authentication Required

Question

I'm trying to open a website (I am behind a corporate proxy) using urllib.request.urlopen() but I am getting the error:

urllib.error.HTTPError: HTTP Error 407: Proxy Authentication Required

I can find the proxy in urllib.request.getproxies(), but how do I specify a username and password to use for it? I couldn't find the solution in the official docs.

Have you seen stackoverflow.com/questions/34079/…? Examples at the bottom of docs.python.org/library/urllib2.html#urllib2-examples. — Katriel
– Katriel, Commented Aug 1, 2012 at 16:01
Yeah, but that's for Python2.7 Didn't they restructure the entire urllib package in Python3 3? — Lanaru
– Lanaru, Commented Aug 1, 2012 at 16:03
They didn't fundamentally change the interface -- just moved things around a bit. ProxyHandler now lives in urllib.request.ProxyHandler — Katriel
– Katriel, Commented Aug 1, 2012 at 16:04

Lanaru · Accepted Answer · 2012-08-01 16:12:22Z

29

import urllib.request as req

proxy = req.ProxyHandler({'http': r'http://username:password@url:port'})
auth = req.HTTPBasicAuthHandler()
opener = req.build_opener(proxy, auth, req.HTTPHandler)
req.install_opener(opener)
conn = req.urlopen('http://google.com')
return_str = conn.read()

answered Aug 1, 2012 at 16:12

Lanaru

9,7917 gold badges43 silver badges66 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

tommy.carstensen Over a year ago

Thanks. Is there no way to do this without supplying username and password?

Jim Dennis Over a year ago

If you're worried about having credentials hard-coded in your source code (and thus leaking into git or other VCS artifacts, and so on) then the best approach is to use something like configparser, or YAML or JSON, to store the credentials in their own separate file. Build the ProxyHandler URL dynamically from the config settings. This allows your sources to be readable while keeping credentials confidential.

mcherm Over a year ago

A minor note: for me, where he has "@url:port" I actually used the machine name "@machine:port", not a full URL.

FancyDolphin Over a year ago

I'm using Python 2.7, how would this look? Would I have to use urllib2?

Bernad Peter · Accepted Answer · 2020-01-29 06:11:05Z

3

You can set the proxy server authentication with your credentials(username and password) to connect to the website using requests. This worked for me. To get your proxy server name: use

import urllib.request as req
import os
#get your proxy server url details using below command
req.getproxies() 

#user your credentials and url to authenticate

os.environ['http_proxy'] = "http://username:pwd@url:80"
os.environ['https_proxy'] = "http://username:pwd@url:80"

#replace username, pwd and url with your credentials.

conn = req.urlopen('https://Google.com')
return_str = conn.read()

edited Jan 29, 2020 at 6:11

answered Jan 20, 2020 at 2:34

Bernad Peter

5145 silver badges12 bronze badges

1 Comment

Simas Joneliunas Over a year ago

It would be great if you could add a bit more comments to explain how the code solves the OP's problem.

Collectives™ on Stack Overflow

Python 3 - urllib, HTTP Error 407: Proxy Authentication Required

2 Answers 2

4 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related