Daemon Threads Explanation

Let's say you're making some kind of dashboard widget. As part of this, you want it to display the unread message count in your email box. So you make a little thread that will:

1. Connect to the mail server and ask how many unread messages you have.
2. Signal the GUI with the updated count.
3. Sleep for a little while.

When your widget starts up, it would create this thread, designate it a daemon, and start it. Because it's a daemon, you don't have to think about it; when your widget exits, the thread will stop automatically.

I will also add my few bits here, I think one of the reasons why daemon threads are confusing to most people(atleast they were to me) is because of the Unix context to the word dameon.

In Unix terminology the word daemon refers to a process which once spawned; keeps running in the background and user can move on to do other stuff with the foreground process.

In Python threading context, every thread upon creation runs in the background, whether it is daemon or non-daemon, the difference comes from the fact how these threads affect the main thread.

When you start a non-daemon thread, it starts running in background and you can perform other stuff, however, your main thread will not exit until all such non-daemon threads have completed their execution, so in a way, your program or main thread is blocked.

With daemon threads they still run in the background but with one key difference that they do not block the main thread. As soon as the main thread completes its execution & the program exits, all the remaining daemon threads will be reaped. This makes them useful for operations which you want to perform in background but want these operations to exit automatically as soon as the main application exits.

One point to keep note of is that you should be aware of what exactly you are doing in daemon threads, the fact they exit when main thread exits can give you unexpected surprises. One of the ways to gracefully clean up the daemon threads is to use the Threading Events to set the event as an exit handler and check if the event is set inside the thread and then break from the thread function accordingly.

Another thing that confused about daemon threads is the definition from python documentation.

The significance of this flag is that the entire Python program exits when only daemon threads are left

In simple words what this means is that if your program has both daemon and non-daemon threads the main program will be blocked and wait until all the non-daemon have exited, as soon as they exit main thread will exit as well. What this statement also implies but is not clear at first glance is that all daemon threads will be exited automatically once the main threads exits.

A simpler way to think about it, perhaps: when main returns, your process will not exit if there are non-daemon threads still running.

A bit of advice: Clean shutdown is easy to get wrong when threads and synchronization are involved - if you can avoid it, do so. Use daemon threads whenever possible.

Other posters gave some examples for situations in which you'd use daemon threads. My recommendation, however, is never to use them.

It's not because they're not useful, but because there are some bad side effects you can experience if you use them. Daemon threads can still execute after the Python runtime starts tearing down things in the main thread, causing some pretty bizarre exceptions.

More info here:

https://joeshaw.org/python-daemon-threads-considered-harmful/

https://mail.python.org/pipermail/python-list/2005-February/343697.html

Strictly speaking you never need them, it just makes implementation easier in some cases.

Quoting Chris: "... when your program quits, any daemon threads are killed automatically.". I think that sums it up. You should be careful when you use them as they abruptly terminate when main program executes to completion.

Chris already explained what daemon threads are, so let's talk about practical usage. Many thread pool implementations use daemon threads for task workers. Workers are threads which execute tasks from task queue.

Worker needs to keep waiting for tasks in task queue indefinitely as they don't know when new task will appear. Thread which assigns tasks (say main thread) only knows when tasks are over. Main thread waits on task queue to get empty and then exits. If workers are user threads i.e. non-daemon, program won't terminate. It will keep waiting for these indefinitely running workers, even though workers aren't doing anything useful. Mark workers daemon threads, and main thread will take care of killing them as soon as it's done handling tasks.

When your second thread is non-Daemon, your application's primary main thread cannot quit because its exit criteria is being tied to the exit also of non-Daemon thread(s). Threads cannot be forcibly killed in python, therefore your app will have to really wait for the non-Daemon thread(s) to exit. If this behavior is not what you want, then set your second thread as daemon so that it won't hold back your application from exiting.

Create a Daemon thread when:

• You want a low-priority thread
• Your Thread does background-specific tasks, and more importantly,
• When you want this thread to die as soon as all user threads accomplish their tasks.

Some Examples of Daemon Thread Services: Garbage collection in Java, Word count checker in MS Word, Auto-saver in medium, File downloads counter in a parallel file downloads application, etc.

In Python, threads can be of two types: daemon and non-daemon (sometimes called ‘normal threads’ or simply ‘threads’). The main difference between these types of threads is how they affect the lifetime of the entire program:

• Daemon Threads: These are considered ‘background’ threads. These threads run in the background and do not prevent the main program from finishing. Once all non-daemon threads in the program have finished, the program will terminate, closing all daemon threads that are still running, regardless of whether they have finished their task or not.

• Non-Daemon Threads: These are the ‘main’ threads of the program. The program will continue to run as long as at least one of these threads is alive. Only when the last non-daemon thread has finished, the program as a whole will be able to terminate.

Summary:

The entire Python program terminates when there are no non-daemon threads left alive. This means that, regardless of how many daemon threads are running, the program will terminate when all non-daemon threads have finished executing. The daemon threads cannot keep the program running on their own; they are intended for background tasks that do not need to stop the program from terminating.

Example:

#!/usr/bin/env python3
""" Barron finishes cooking while Olivia cleans """

import threading
import time

def kitchen_cleaner():
    while True:
        print('Olivia cleaned the kitchen.')
        time.sleep(1)

if __name__ == '__main__':
    olivia = threading.Thread(target=kitchen_cleaner)
    olivia.daemon = True
    olivia.start()

    print('Barron is cooking...')
    time.sleep(0.6)
    print('Barron is cooking...')
    time.sleep(0.6)
    print('Barron is cooking...')
    time.sleep(0.6)
    print('Barron is done!')

Even though this question has already 12 answers and @Rohit's answer explained the situation very well, when I check the server sample code in https://docs.python.org/3/library/socketserver.html I still feel a bit confused.

if __name__ == "__main__":
    # Port 0 means to select an arbitrary unused port
    HOST, PORT = "localhost", 0

    server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
    with server:
        ip, port = server.server_address

        # Start a thread with the server -- that thread will then start one
        # more thread for each request
        server_thread = threading.Thread(target=server.serve_forever)
        # Exit the server thread when the main thread terminates
        server_thread.daemon = True
        server_thread.start()

The comment there said by setting server_thread.daemon = True I can then "exit the server thread when the main thread terminates".

That is the situation I faced, I have a GUI program using a thread run a HTTPServer. I didn't set the daemon property at the first because I had though if I set that, the HTTPServer thread would continue to run in background even after the parent thread (my GUI thread) exits. Then it turned out to be the opposite that even after I kill my GUI program, the port my http server used was still in the used state so I would fail to restart my program.