Python parallel threads

Question

Here the code which download 3 files, and do something with it. But before starting Thread2 it waits until Thread1 will be finished. How make them run together? Specify some examples with commentary. Thanks

import threading
import urllib.request


def testForThread1():
    print('[Thread1]::Started')
    resp = urllib.request.urlopen('http://192.168.85.16/SOME_FILE')
    data = resp.read()
    # Do something with it
    return 'ok'


def testForThread2():
    print('[Thread2]::Started')
    resp = urllib.request.urlopen('http://192.168.85.10/SOME_FILE')
    data = resp.read()
    # Do something with it
    return 'ok'


if __name__ == "__main__":
    t1 = threading.Thread(name="Hello1", target=testForThread1())
    t1.start()
    t2 = threading.Thread(name="Hello2", target=testForThread2())
    t2.start()
    print(threading.enumerate())
    t1.join()
    t2.join()
    exit(0)

You're executing the thread-target function in the thread assignment. Effectively, you are executing threading.Thread(name='Hello1', target='ok'). Try t1 = threading.Thread(name="Hello1", target=testForThread1) instead. — Nisan.H
– Nisan.H, Commented Sep 3, 2013 at 21:51

Nisan.H · Accepted Answer · 2013-09-03 21:56:36Z

You are executing the target function for the thread in the thread instance creation.

if __name__ == "__main__":
    t1 = threading.Thread(name="Hello1", target=testForThread1()) # <<-- here
    t1.start()

This is equivalent to:

if __name__ == "__main__":
    result = testForThread1() # == 'ok', this is the blocking execution
    t1 = threading.Thread(name="Hello1", target=result) 
    t1.start()

It's Thread.start()'s job to execute that function and store its result somewhere for you to reclaim. As you can see, the previous format was executing the blocking function in the main thread, preventing you from being able to parallelize (e.g. it would have to finish that function execution before getting to the line where it calls the second function).

The proper way to set the thread in a non-blocking fashion would be:

if __name__ == "__main__":
    t1 = threading.Thread(name="Hello1", target=testForThread1) # tell thread what the target function is
    # notice no function call braces for the function "testForThread1"
    t1.start() # tell the thread to execute the target function

johannchopin · Accepted Answer · 2021-01-10 16:12:09Z

3

For this, we can use threading but it's not efficient since you want to download files. so the total time will be equal to the sum of download time of all files. If you have good internet speed, then multiprocessing is the best way.

import multiprocessing


def test_function():
    for i in range(199999998):
        pass
    
    
t1 = multiprocessing.Process(target=test_function)
t2 = multiprocessing.Process(target=test_function)
t1.start()
t2.start()

This is the fastest solution. You can check this using following command:

time python3 filename.py

you will get the following output like this:

real    0m6.183s
user    0m12.277s
sys     0m0.009s

here, real = user + sys

user time is the time taken by python file to execute. but you can see that above formula doesn't satisfy because each function takes approx 6.14. But due to multiprocessing, both take 6.18 seconds and reduced total time by multiprocessing in parallel.

You can get more about it from here.

edited Jan 10, 2021 at 16:12

johannchopin

15k11 gold badges64 silver badges125 bronze badges

answered Dec 28, 2017 at 12:03

user7159290

1 Comment

at54321 Over a year ago

In Python (and to be more specific, the CPython implementation), multiprocessing is usually the way to go if CPU is the bottleneck (as is the case with your test_function()). But if the work is IO-bound, threads can do just fine (even better, in fact, as they are more-lightweight).

jinglebird · Accepted Answer · 2024-07-18 10:34:20Z

To make Thread1 and Thread2 run together, you need to ensure that both threads are started without waiting for each other to finish

In your code,functions for the threads are called immediately (target=testForThread1()) instead of being passed as references (target=testForThread1). This causes the functions to execute before the threads are started, That is not you want that

Check bellow code, I think you can run this as parralel thread:

import threading
import urllib.request

def testForThread1():
    //

def testForThread2():
    //

if __name__ == "__main__":
    # Create thread objects with the target functions
    t1 = threading.Thread(name="Hello1", target=testForThread1)
    t2 = threading.Thread(name="Hello2", target=testForThread2)
    
    # Start both threads
    t1.start()
    t2.start()
    
    # List all threads currently running
    print(threading.enumerate())
    
    # Wait for both threads to complete
    t1.join()
    t2.join()
    
    exit(0)

Collectives™ on Stack Overflow

Python parallel threads

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related