Programmatically exiting python script while multithreading

Question

I have some code which runs routinely, and every now and then (like once a month) the program seems to hang somewhere and I'm not sure where.

I thought I would implement [what has turned out to be not quite] a "quick fix" of checking how long the program has been running for. I decided to use multithreading to call the function, and then while it is running, check the time.

For example:

import datetime
import threading

def myfunc():
  #Code goes here

t=threading.Thread(target=myfunc)
t.start()
d1=datetime.datetime.utcnow()
while threading.active_count()>1:
  if (datetime.datetime.utcnow()-d1).total_seconds()>60:
    print 'Exiting!'
    raise SystemExit(0)

However, this does not close the other thread (myfunc).

What is the best way to go about killing the other thread?

If you want other threads to die immediately when SystemExit is raised, make them daemon threads. — roippi
– roippi, Commented Sep 23, 2013 at 3:46

Tim Peters · Accepted Answer · 2013-09-23 03:59:05Z

3

The docs could be clearer about this. Raising SystemExit tells the interpreter to quit, but "normal" exit processing is still done. Part of normal exit processing is .join()-ing all active non-daemon threads. But your rogue thread never ends, so exit processing waits forever to join it.

As @roippi said, you can do

t.daemon = True

before starting it. Normal exit processing does not wait for daemon threads. Your OS should kill them then when the main process exits.

Another alternative:

import os
os._exit(13)  # whatever exit code you want goes there

That stops the interpreter "immediately", and skips all normal exit processing.

Pick your poison ;-)

answered Sep 23, 2013 at 3:59

Tim Peters

71.4k14 gold badges133 silver badges140 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mariusz Jamro Over a year ago

Be aware that this breaks 'atexit' callbacks

Community · Accepted Answer · 2017-05-23 12:11:09Z

There is no way to kill a thread. You must kill the target from within the target. The best way is with a hook and a queue. It goes something like this.

import Threading
from Queue import Queue

# add a kill_hook arg to your function, kill_hook
# is a queue used to pass messages to the main thread
def myfunc(*args, **kwargs, kill_hook=None):
  #Code goes here
  # put this somewhere which is periodically checked.
  # an ideal place to check the hook is when logging
  try:
    if q.get_nowait():  # or use q.get(True, 5) to wait a longer
      print 'Exiting!'
      raise SystemExit(0)
    except Queue.empty:
      pass

q = Queue()  # the queue used to pass the kill call
t=threading.Thread(target=myfunc, args = q)
t.start()
d1=datetime.datetime.utcnow()
while threading.active_count()>1:        
  if (datetime.datetime.utcnow()-d1).total_seconds()>60:
  # if your kill criteria are met, put something in the queue
    q.put(1)

I originally found this answer somewhere online, possibly this. Hope this helps!

Another solution would be to use a separate instance of Python, and monitor the other Python thread, killing it from the system level, with psutils.

Wow, I like the daemon and stealth os._exit solutions too!

Collectives™ on Stack Overflow

Programmatically exiting python script while multithreading

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related