Python Threading with Selenium

Question

I'm attempting to use Splinter (a package for Selenium) in multiple instances. However, instances aren't starting until the first thread completely finishes. So, the browser instance opens up, loads the page, sleeps, and only then does the 2nd thread start.

I need to run instances at the same time.

import threading
import time
from splinter import Browser


def worker(proxy, port):
    proxy_settings = {"network.proxy.type": 1,
                      "network.proxy.ssl": proxy,
                      "network.proxy.ssl_port": port,
                      "network.proxy.socks": proxy,
                      "network.proxy.socks_port": port,
                      "network.proxy.socks_remote_dns": True,
                      "network.proxy.ftp": proxy,
                      "network.proxy.ftp_port": port
                      }

    browser = Browser('firefox',
                      profile_preferences=proxy_settings,
                      capabilities={'pageLoadStrategy': 'eager'}) #eager or normal
    print("Proxy: ", proxy, ":", proxy)
    browser.visit("https://mxtoolbox.com/whatismyip/?" + proxy)
    time.sleep(2)


ip1 = '22.22.222.222'
ip2 = '222.222.22.222'
p1 = int(2222)
p2 = int(2222)

p = []
p.append((ip1,p1))
p.append((ip2,p2))
x = 0
for pp in p:
    threading.Thread(target=worker(pp[0], pp[1])).start()

In my longer code (the above is my attempt to figure out why I can't multi-thread) Im also getting an error in my editor that

Local variable 'browser' value is not used

Adonis · Accepted Answer · 2018-03-08 16:31:52Z

1

That's because you're not starting a thread with the worker function, actually the last line should rather look like:

threading.Thread(target=worker, args=(pp[0], pp[1])).start()

As for your editor issue, I'd say it is editor dependent and without any information, it's hard to say (I'd mention that running pylint does not indicate such warning)

edited Mar 8, 2018 at 16:31

answered Mar 8, 2018 at 12:27

Adonis

4,8183 gold badges41 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python Threading with Selenium

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related