how to Implement proxy in python

Question

I want to implement proxy in my code snippet. How can I achieve this?

try:
     raw = self.session.get(self.BASE_URL + '/archive').text
except:
     logging.info('Error with pastebin')
     raw = None
     sleep(5)
results = BeautifulSoup(raw).find_all(lambda tag: tag.name == 'td' and tag.a and '/archive/' not in tag.a['href'] and tag.a['href'][1:])

Did you just paste some random part of your code? What does your snippet have to do with a proxy? — Burhan Khalid
– Burhan Khalid, Commented Jun 24, 2015 at 7:30

user5043534 · Accepted Answer · 2015-06-24 07:54:07Z

1

You can import urllib and try doing as below to implement proxy

try:
    raw = self.session.get(self.BASE_URL + '/archive').text

except:
    proxy_support = urllib2.ProxyHandler({"http":"http://10.62.51.201:3128"})
    opener = urllib2.build_opener(proxy_support)
            urllib2.install_opener(opener)

    #results = BeautifulSoup(helper.download(urllib2.urlopen(self.BASE_URL).geturl())+  '/archive').find_all(lambda tag: tag.name == 'td' and tag.a and '/archive/' not in tag.a['href'] and tag.a['href'][1:])
    myurl = urllib2.urlopen(self.BASE_URL).geturl()
    raw = self.session.get(myurl + '/archive').text

results = BeautifulSoup(raw).find_all(lambda tag: tag.name == 'td' and tag.a and '/archive/' not in tag.a['href'] and tag.a['href'][1:])

answered Jun 24, 2015 at 7:54

user5043534

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

how to Implement proxy in python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related