Multiprocessing synchronization between multiple processes (Python)

Question

So there in college practical class was an exercise:

"Create program that defines shared value as 0, then creates and starts 500 processes, each of which increases value of shared value by 1, and finally in the end prints shared value. Run this code few times and see what happens, explain...."

Final version looks like:

from multiprocessing import Process, Value


n=Value('i', 0)

def fun():
        n.value+=1

for i in range(500):
        p=Process(target=fun).start()

print n.value

Output value varies in range 420-480 and i understand why.
The questions is how to make it always 500 if it possible? I've been reading Python docs and have found possible solution - Semaphore, but maybe I didn't understand it well..
In the case of Semaphore code looks:

from multiprocessing import Process, Value, Semaphore

n=Value('i', 0)
sem=Semaphore()

def fun():
        sem.acquire()
        n.value+=1
        sem.release()

for i in range(500):
    p=Process(target=fun).start()
print n.value

In this case the final output varies in range 492-495 - way better.

P.S. Do not advise me to use threads from threading class - question is about multiprocessing.

Володимир Кельбель · Accepted Answer · 2017-10-30 11:52:09Z

2

Your processes are not joined. Your lock therefore works, but the value of n is displayed before all of the processes are done to increment it, hence n<500. You can try the following code:

from multiprocessing import Process, Value, Semaphore

n=Value('i', 0)
sem=Semaphore()

def fun():
        sem.acquire()
        n.value+=1
        sem.release()

pcs=[]
for i in range(500):
    p=Process(target=fun)
    pcs.append(p)
    p.start()
for i in pcs:
    i.join()
print n.value

edited Oct 30, 2017 at 11:52

Володимир Кельбель

851 silver badge12 bronze badges

answered Oct 29, 2017 at 23:40

Abel Riboulot

1882 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Володимир Кельбель Over a year ago

Your code works like a first version, and if i define Lock outside loop, before defining functions, it works just like semaphore, because semaphore is a modified lock

Abel Riboulot Over a year ago

Hey, my bad, the issue seems to actually be that the processes are not being joined, please check out the edited code, I ran it and it seems to work fine. The semaphore you setup works well.

Володимир Кельбель Over a year ago

In your edited code is no sense for me.. What i want is to execute all processes parallel, and if i join every process after it start it just works like normal loop 'for i in range(500)' - every process will not start before previous didn't end, and you do not need any semaphore or lock for this. For example if you put time.sleep(1) inside function - you will need to wait 500 seconds till it end.

Abel Riboulot Over a year ago

Yeah sorry, I got the indentation wrong. Basically, put the join outside the loop. Now all processes are started, and joined before the value is printed. The original issue was that it did not wait for every processes to be finished in order to print the output, it just waited for it to be started. The last few processes would not be finished before printing the output. Adding "p.join()" allows to wait for all processes to be finished before printing it.

Володимир Кельбель Over a year ago

will try it tomorrow, ty btw!

|

Collectives™ on Stack Overflow

Multiprocessing synchronization between multiple processes (Python)

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related