31

I'm trying to create two threads that each have their own asyncio event loop.

I've tried the following code but it doesn't seem to work:

import asyncio
from threading import Thread

def hello(thread_name):
    print('hello from thread {}!'.format(thread_name))

event_loop_a = asyncio.new_event_loop()
event_loop_b = asyncio.new_event_loop()

def callback_a():
    asyncio.set_event_loop(event_loop_a)
    asyncio.get_event_loop().call_soon_threadsafe(lambda: hello('a'))

def callback_b():
    asyncio.set_event_loop(event_loop_b)
    asyncio.get_event_loop().call_soon_threadsafe(lambda: hello('b'))

thread_a = Thread(target=callback_a, daemon=True)
thread_b = Thread(target=callback_b, daemon=True)
thread_a.start()
thread_b.start()

My use case is calling Tornado web framework's websocket_connect async function.

Improve this question
2
  • 2
    What does "it doesn't seem to work" mean? Are you getting an exception? Are the event loops not running? Commented Sep 12, 2018 at 15:53
  • the event loops aren't running, no exceptions but the event loop doesn't seem to be set in the thread Commented Sep 12, 2018 at 16:29

1 Answer 1

Reset to default
29

Your threads are queuing callbacks in their respective event loops, but they exit before actually running the event loop, so the callbacks never get executed. Also, you don't need call_soon_threadsafe, since you are invoking the callback from the same thread the event loop is (or, rather, will be) running on.

This code prints the expected output:

import asyncio
from threading import Thread

def hello(thread_name):
    print('hello from thread {}!'.format(thread_name))

event_loop_a = asyncio.new_event_loop()
event_loop_b = asyncio.new_event_loop()

def callback_a():
    asyncio.set_event_loop(event_loop_a)
    asyncio.get_event_loop().call_soon(lambda: hello('a'))
    event_loop_a.run_forever()

def callback_b():
    asyncio.set_event_loop(event_loop_b)
    asyncio.get_event_loop().call_soon(lambda: hello('b'))
    event_loop_b.run_forever()

thread_a = Thread(target=callback_a, daemon=True)
thread_b = Thread(target=callback_b, daemon=True)
thread_a.start()
thread_b.start()

A more typical use case for call_soon_threadsafe, and more in line with what you might have had in mind, is to submit a callback (or a coroutine using asyncio.run_coroutine_threadsafe) to an event loop running in another thread. Here is an example:

import asyncio, threading

def hello(thread_name):
    print('hello from thread {}!'.format(thread_name))

event_loop_a = asyncio.new_event_loop()
event_loop_b = asyncio.new_event_loop()

def run_loop(loop):
    asyncio.set_event_loop(loop)
    loop.run_forever()

threading.Thread(target=lambda: run_loop(event_loop_a)).start()
threading.Thread(target=lambda: run_loop(event_loop_b)).start()

event_loop_a.call_soon_threadsafe(lambda: hello('a'))
event_loop_b.call_soon_threadsafe(lambda: hello('b'))

event_loop_a.call_soon_threadsafe(event_loop_a.stop)
event_loop_b.call_soon_threadsafe(event_loop_b.stop)

In that case it is rarely needed to have more than one event loop thread - you'd typically create only one, and allow that thread to service all your asyncio needs. After all, being able to host a large number of tasks in a single event loop is one of the strong points of asyncio.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

This answer helped me get Tornado websocket_connect work correctly in its own thread with its own eventloop, thanks for this answer! The important piece I was missing was loop.run_forever(), and the call_soon_threadsafe helped me schedule the functions I wanted. Thanks!
@RudolfOlah If you like call_soon_threadsafe, also check out asyncio.run_coroutine_threadsafe, which allows you to schedule a coroutine in the event loop thread, and returns a concurrent.futures.Future which allows waiting for the coroutine to finish without causing a deadlock.
thanks for the tip; in my other case, with Tornado websocket_connect, the future finishes upon connection: tornadoweb.org/en/stable/… I guess it would make sense for the loop to run that function until completion (websocket client has connected to the server successfully), and then run the loop forever.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.