Hai I want to use private proxy in python. Please help me to add username,password of proxy in my current Python script.I know there is few lines to add to get in run successfully.
data = urllib.urlencode(values)
proxy_support = urllib2.ProxyHandler({"http": httpproxy})
opener = urllib2.build_opener(proxy_support)
urllib2.install_opener(opener)
req = urllib2.Request(url)
response = urllib2.urlopen(req,data)
page = response.read()
Thanks in advance!
Outdated, but I found a site which might resolve this problem.
http://code.activestate.com/recipes/523016-using-xmlrpc-with-authenticated-proxy-server/
The relevent code: import base64 import urllib from urllib import unquote, splittype, splithost import xmlrpclib
class UrllibTransport(xmlrpclib.Transport):
def set_proxy(self, proxy):
self.proxyurl = proxy
def request(self, host, handler, request_body, verbose=0):
type, r_type = splittype(self.proxyurl)
phost, XXX = splithost(r_type)
puser_pass = None
if '@' in phost:
user_pass, phost = phost.split('@', 1)
if ':' in user_pass:
user, password = user_pass.split(':', 1)
puser_pass = base64.encodestring('%s:%s' % (unquote(user),
unquote(password))).strip()
urlopener = urllib.FancyURLopener({'http':'http://%s'%phost})
if not puser_pass:
urlopener.addheaders = [('User-agent', self.user_agent)]
else:
urlopener.addheaders = [('User-agent', self.user_agent),
('Proxy-authorization', 'Basic ' + puser_pass) ]
host = unquote(host)
f = urlopener.open("http://%s%s"%(host,handler), request_body)
self.verbose = verbose
return self.parse_response(f)
if __name__ == '__main__':
proxy = "http://proxy_user:[email protected]:8080"
p = UrllibTransport()
p.set_proxy(proxy)
It creates a urllib2 transport class that includes authentication.
urrlib, not urrlib2, which is more functional. In urrlib2 you don't have to implement user-agent and stuff handling by hand. The only reason I know for urllib usage nowdays is urllib.urlencode, missed in urllib2What is value of your httpproxy variable? Did you try
httpproxy = 'http://username:password@proxyurl:proxyport'
By the way, urrlib2 seamlessly handles http_proxy environment variable for proxy configuration and no_proxy for proxy exemptions. For example for *nix platforms following code should work:
export http_proxy=http://username:password@proxyurl:proxyport
