threads not running parallel in python script

Question

I am new to python and threading. I am trying to run multiple threads at a time. Here is my basic code :

  import threading
  import time
  threads = []

  print "hello"

  class myThread(threading.Thread):
          def __init__(self,i):
                  threading.Thread.__init__(self)
                  print "i = ",i
                  for j in range(0,i):
                          print "j = ",j
                          time.sleep(5)

  for i in range(1,4):
          thread = myThread(i)
          thread.start()

While 1 thread is waiting for time.sleep(5) i want another thread to start. In short, all the threads should run parallel.

Afaik, threads in python all run on a single core. So the only thing threads can do is increase the cpu burst so to speak. You need the multiprocessing library that starts new processes to get more than a single core. — willeM_ Van Onsem
– willeM_ Van Onsem, Commented Feb 2, 2017 at 12:00
when the thread thread is constructed, it is on the same thread. If you want your code to be run concurently, you should put your time.sleep' in the run` method, not the constructor. — n00dl3
– n00dl3, Commented Feb 2, 2017 at 12:04
You may want to look at this answer on how to use threading. You basically don't need to subclass Thread at all, but if you do, don't use the __init__() constructor to perform tasks that should be threaded. — ImportanceOfBeingErnest
– ImportanceOfBeingErnest, Commented Feb 2, 2017 at 12:07
No, in the code you show, everything is sequential, because you perform your work in the constructor of the Thread, even before the thread is started. — ImportanceOfBeingErnest
– ImportanceOfBeingErnest, Commented Feb 2, 2017 at 12:11

Shane · Accepted Answer · 2017-02-02 12:24:28Z

7

You might have some misunderstandings on how to subclass threading.Thread, first of all __init__() method is roughly what represents a constructor in Python, basically it'll get executed every time you create an instance, so in your case when thread = myThread(i) executes, it'll block till the end of __init__().

Then you should move your activity into run(), so that when start() is called, the thread will start to run. For example:

import threading
import time
threads = []

print "hello"

class myThread(threading.Thread):
    def __init__(self, i):
        threading.Thread.__init__(self)
        self.i = i

    def run(self):
        print "i = ", self.i
        for j in range(0, self.i):
            print "j = ",j
            time.sleep(5)

for i in range(1,4):
    thread = myThread(i)
    thread.start()

P.S. Because of the existence of GIL in CPython, you might not be able to fully take advantages of all your processors if the task is CPU-bound.

answered Feb 2, 2017 at 12:24

Shane

2,4011 gold badge16 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Utkarsh Patel Over a year ago

It's working and i am finally understanding how it works. Thanks a ton.

ImportanceOfBeingErnest · Accepted Answer · 2017-02-02 12:22:35Z

3

Here is an example on how you could use threading based on your code:

import threading
import time
threads = []

print "hello"

def doWork(i):
    print "i = ",i
    for j in range(0,i):
        print "j = ",j
        time.sleep(5)

for i in range(1,4):
    thread = threading.Thread(target=doWork, args=(i,))
    threads.append(thread)
    thread.start()

# you need to wait for the threads to finish
for thread in threads:
    thread.join()

print "Finished"

answered Feb 2, 2017 at 12:22

ImportanceOfBeingErnest

342k61 gold badges737 silver badges771 bronze badges

Comments

LouisXW · Accepted Answer · 2019-03-17 15:48:56Z

2

import threading
import subprocess


def obj_func(simid):
    simid = simid
    workingdir = './' +str (simid) # the working directory for the simulation
    cmd = './run_delwaq.sh' # cmd is a bash commend to launch the external execution
    subprocess.Popen(cmd, cwd=workingdir).wait()


def example_subprocess_files():
    num_threads = 4
    jobs = []

    # Launch the threads and give them access to the objective function
    for i in range(num_threads):
        workertask = threading.Thread(target=obj_func(i))
        jobs.append(workertask)

    for j in jobs:
        j.start()

    for j in jobs:
        j.join()

    print('All the work finished!')


if __name__ == '__main__':
    example_subprocess_files()

This one not works for my case that the task is not print but CPU-Intensive task. The thread are excluded in serial.

edited Mar 17, 2019 at 15:48

answered Mar 17, 2019 at 2:58

LouisXW

514 bronze badges

3 Comments

Anantha Raju C Over a year ago

for better understanding provide an explanation to your solution

LouisXW Over a year ago

I have updated the code above to the one that I used in my case. For each thread, the worker need to run a external program. I create separate folder for each external program so they are not interact with each other. However, when I check the result I found the external program in each thread is running in serial instead of in parallel.

LouisXW Over a year ago

I tested the solution posted by @Shane for my problem(that overrides the run() method in a subclass of Threading.Thread class) . It works. However, not quite sure why the alternative method I posted above not work. Since according to the document in Threading.Thread, both method should work.

Collectives™ on Stack Overflow

threads not running parallel in python script

3 Answers 3

1 Comment

Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related