3

I'm using a semaphore to hand work off to another thread in Python. I.e. the master thread will put an item on a queue, then call the semaphore's release method; the worker thread will call acquire on the semaphore, then pop an item off the queue to work on.

There's also a special TERMINATE item that the master can put on the queue, which instructs the worker to end. My question is, should the worker aim to issue acquires to match any outstanding releases on the semaphore, before it terminates? The semaphore belongs to the worker object and is not used again after the thread terminates; the process however may be long-lived and create many worker objects/threads/semaphores in future.

  1. Is it safe to abandon the semaphore with a non-zero count? (I suspect it is but I want to double check.)

  2. Regardless of whether it's safe, do you (subjectively) think it's "nicer" to fully clean up the semaphore before ending?

I'm mostly interested in the semaphore behaviour in Python (and more specifically CPython). However any general wisdom on semaphores in other languages, such as C's pthreads, would be welcome too.

Improve this question
5
  • 1
    I realize this isn't an answer to your question, but it sounds like you might benefit from Python's synchronized Queue class, designed specifically for this type of situation. Commented Sep 6, 2011 at 0:38
  • "should the worker aim to issue acquires to match any outstanding releases on the semaphore?": There will be items after the TERMINATE item? Commented Sep 6, 2011 at 0:41
  • 1
    Queue does look useful so I'll probably rewrite to use it. Though I have welcomed the multithreading exercise of writing my own implementation as a first draft... Commented Sep 6, 2011 at 0:44
  • @Nayuki - there may be; the worker is allowed to ignore any items after the TERMINATE. Commented Sep 6, 2011 at 0:46
  • If you like scalable designs, AMQP is worth a look, try Celery (celeryproject.org). It scales way better than thread/semaphores because producer and consumers can run on distinct machines. Commented Sep 6, 2011 at 1:27

1 Answer 1

Reset to default
2

#1: Yes, it's safe to abandon the semaphore with a non-zero count. It's just a value after all.

#2: It's nicer to reduce the amount of code. Strive to write the minimum amount of clear code for a correct implementation.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Is it just a value? Semaphore objects in Python have associated Conditions, Locks, lists of waiters, etc... I wanted to check that all that stuff is safe to abandon too. I wasn't sure whether the implementation details might be using other resources that may not get cleaned up properly when the count is left >0.
Well, the internal bookkeeping stuff associated with the semaphore shouldn't be dependent on the semaphore's value. I am pretty confident that no "cleanup to zero" is needed.
I think you're right. After all, we can create a semaphore with an initial value of non-zero and there's no discussion of whether the intention is to regress to the inital value at the end of the semaphore's lifetime.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.