Await inside ctypes callback with loop already running

Question

I need to perform a series of read/write tasks inside a ctypes callback function, but still inside an async task that is responsible for giving such read/write tasks:

async def main(ble_address):
    async with BleakClient(ble_address) as client:
        def my_io_callback(...)
            # await client.read_gatt_char(UUID) ???
            # await client.write_gatt_char(UUID, ...) ???
        my_c_function(to_c_function(my_io_callback))

asyncio.run(main(ble_address))

I can't move the whole async block inside the callback because it will be called several times and I need the device to be connected during the entire interaction.

What's the proper way of dealing with this situation? Answers I've seen so far haven't quite cover this particular case.

You can't use an await expression in a sync function, but you can create a Task. Can you put your async code into a new async def function, and create a Task from that function inside my_io_callback? — Paul Cornelius
– Paul Cornelius, Commented Dec 6, 2024 at 2:52
@PaulCornelius maybe I could, but then how do I wait for the task completion inside my_io_callback without using await? — Caian
– Caian, Commented Dec 6, 2024 at 9:32
You can't await Task completion inside your callback. But you can set a second callback to run when the Task is complete (see the function Task.add_done_callback). — Paul Cornelius
– Paul Cornelius, Commented Dec 7, 2024 at 3:23
Unfortunately this is impossible because the c callback (third party library) is supposed to block until the end of the IO operation. — Caian
– Caian, Commented Dec 7, 2024 at 10:02
I "solved" it by moving all async code to a separate thread and using traditional synchronization primitives (Lock / CVars) to exchange IO requests and results. Frankly it is an overly complicated solution for something that could be a non-await wait-for-task. — Caian
– Caian, Commented Dec 7, 2024 at 10:06

Booboo · Accepted Answer · 2024-12-07 14:44:21Z

Without knowing the fine details of your code it is possible that the following is a solution:

I would call an async function from a sync functions using the run_coro function below. The first time this function is called, it creates a new daemon thread running an event loop and then uses asyncio.run_coroutine_threadsafe to run the async function in the new thread:

import asyncio
import threading

_event_loop = None

def run_async(coro):
    """await a coroutine from a synchronous function/method."""

    global _event_loop

    # Lazily create a new event loop and a daemon thread that will run it:
    if not _event_loop:
        _event_loop = asyncio.new_event_loop()
        threading.Thread(target=_event_loop.run_forever, name="Async Runner", daemon=True).start()

    return asyncio.run_coroutine_threadsafe(coro, _event_loop).result()

async def main():
    sync_function()  # call a sync function

def sync_function():
    # We want to await an async_function, so:
    result = run_async(async_function())
    print(result)

async def async_function():
    await asyncio.sleep(1)
    return 7

asyncio.run(main())

Prints:

Note that the call to run_coro will block the main thread until the async function being invoked completes. But you stated in one of your comments:

Unfortunately this is impossible because the c callback (third party library) is supposed to block until the end of the IO operation.

So this blocking behavior should be exactly what you want.

I like this idea, is way better than relying on synchronization primitives like what I was doing.

jsbueno · Accepted Answer · 2024-12-07 14:42:22Z

0

Like Paul puts it i the comments: from a sync function you can create a task = just take note of it in a channel where your code outside the c scope can await for it -

Or you might, create a task and use the .add_done_callback to get a synchronous function to be called when the task is resolved.

...


all_tasks = set()

async def result_checker():
    # continous loop fetching the results of the async
    # tasks created inside the c-types callback
    while True:
        for next_task in asyncio.as_completed(all_tasks):
            result = await next_task
            ...



async def main(ble_address):
    result_checker_task = asyncio.create_task(result_checker())
    async with BleakClient(ble_address) as client:
        def my_io_callback(...)
            t1 = asyncio.create_task(client.read_gatt_char(UUID))
            # if you need the result of t1 inside this function:
            def inner(t1_task):
                result = t1_task.result()
                ...
            t1.add_done_callback(inner)
            # await client.write_gatt_char(UUID, ...) ???
        my_c_function(to_c_function(my_io_callback))
    ...
    # do things
    # at end:
    result_checker_task.cancel()

asyncio.run(main(ble_address))

answered Dec 7, 2024 at 14:42

jsbueno

113k11 gold badges159 silver badges239 bronze badges

4 Comments

Booboo Dec 7, 2024 at 14:57

Looking at the OP's code, it appears to me that in the callback when the coroutine client.read_gatt_char(UUID) completes it then wants to call client.write_gatt_char(UUID, ...) . But you have commented out this second call. So how would you ensure in my_io_callback that this write call is made after the read call completes? Shouldn't you create a new async function that contains both the write and read requests and it is this new function that should be the argument to asyncio.create_task in your callnack? (more...)

Booboo Dec 7, 2024 at 14:59

If I understand a comment made by the OP, they want the main thread's event loop to block until the callback completes. But your approach does not permit that. Of course, I may be missing something as far as what the OP is saying.

Caian Dec 7, 2024 at 22:31

@Booboo is right, the C library itself is synchronous, but Bleak uses asyncio. I must be inside of an asynchronous region because of Bleak's connection, can´t use the async for the C callback function, and must wait for IO (without using await because I'm not inside an async function)

jsbueno Dec 8, 2024 at 18:20

the way to go them is to make the async loop run in a different thread, submit the async call to that thread instead, and just go on a semi-busy loop (while + sync time.sleep call) checking when it is done in the ctypes-sync thread.} Either that or arrange a way for things to work through callbacks, but it might not be feasible at all.

Collectives™ on Stack Overflow

Await inside ctypes callback with loop already running

2 Answers 2

1 Comment

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related