Python Multithreading Queue

Question

import threading
from queue import Queue



print_lock = threading.Lock()
def job(worker):
    with print_lock:
        with open('messages.txt') as f:
            for line in f:
                print(line)

def reader():
    while True:
        worker = q.get()
        job(worker)
        q.task_done()

q = Queue()

for x in range(10):
    t = threading.Thread(target=reader)

    t.daemon = True
    t.start()


for worker in range(1):
    q.put(worker)

q.join()

So what i want is each thread reads different messages,

yixing yan · Accepted Answer · 2020-09-23 15:04:07Z

5

Queue is thread safe

so, threadling lock does not needed

edited Sep 23, 2020 at 15:04

answered Aug 2, 2020 at 15:41

yixing yan

1831 silver badge7 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Dinesh Pundkar · Accepted Answer · 2017-07-28 10:25:19Z

2

your trying too many things to learn in same code snippet like 1) Multi-Threading 2) Queue Data Structure 3) Thread Synchronization Mechanisms 4) Locking etc.

Let me answer regarding multi-threading only.

In your case, every thread is reading all messages because target function "job" is opening file and reading all the data and every thread is calling that target function.

Let me simplify the stuff bit.

You want to read each line of file in new thread. So, instead of opening file in every thread and read it, we will open file one time and put data in list.
Now, every thread will read one line from list and print it. Also, it will remove that printed line from list.
Once, all the data is printed and still thread trying to read, we will add the exception.

Code :

import threading
import sys


#Global variable list for reading file data
global file_data
file_data = []

#Create lock
lock = threading.Lock()


def reader():
    while len(file_data) != 0:
        print threading.currentThread().getName() + " --- "

        try:
            lock.acquire()
            #Get one line from list and print it
            a = file_data.pop()
            print a

        except:
            #Once data is not present, let's print exception message
            print "------------------------"
            print "No data present in file"
            sys.exit() 
        lock.release()

#Read data from file and put it into list
with open("messages.txt") as fh:
    file_data = fh.readlines()

for x in range(2):
    name = "Thread_"+str(x)
    t = threading.Thread(name=name,target=reader)
    t.start()

Output:

    C:\Users\dinesh\Desktop>python demo.py
Thread_0 --- Thread_1 ---
Each thread read each message

Thread_1 --- I am great


Thread_0 --- How Are you ?


Thread_1 --- Grey


Thread_0 --- Hey


Thread_1 --- Dinesh


Thread_0 --- Hello


------------------------
No data present in file

C:\Users\dinesh\Desktop>

    C:\Users\dinesh\Desktop>

NOTE : I know sue of global is not recommended. But for learning purpose it is good.

edited Jul 28, 2017 at 10:25

answered Jul 28, 2017 at 9:25

Dinesh Pundkar

4,1962 gold badges26 silver badges38 bronze badges

3 Comments

Grey Byte Over a year ago

Okay i almost got it But the problem with your code is , if i need to read 10 lines i need to create 10 threads what am looking for the soloution even if i have 10 message and 2 threads , they need to read that 10 messages

Grey Byte Over a year ago

Okay thanks a lot for helping me but still i didn't got a perfect answer With your updated code , Only one thread reading all the message This is the output i got while adding a thread name hey Thread-1 hey Thread-1 hey Thread-1 yo Thread-1 hey Thread-1 hello Thread-1 So i think thread 1 is reading all the message Thanks again

Dinesh Pundkar Over a year ago

Please check my updated code now. I have also added Thread name. But I am getting both threads accessing parallely.

Collectives™ on Stack Overflow

Python Multithreading Queue

2 Answers 2

Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related