calling sync functions from async function

Question

I am in the process of trying to port a flask app to quart to utilise asyncio. I don't think my current approach is working, as my entire function chain is written without async in mind - consider the following:

def long_running_task(task):
    result = some_synchronous_function(task)
    return result

@app.route('/<task>', methods=['GET'])
async def do_task(task):
    ok = await long_running_task(task)
    if ok:
        return (ok.result)
    else:
        return ('Something went wrong')

If long_running_task and its whole chain of function calls are not declared as async, am I actually getting any benefit from my route being declared as async?

Nope, that shouldn't even work. You can only await awaitables — async functions etc. To use the power of async, you need to transform your I/O (DB access, etc.) to asynchronous access. — L3viathan
– L3viathan, Commented Feb 14, 2019 at 7:34

J. Taylor · Accepted Answer · 2019-02-14 09:51:27Z

32

To run a blocking synchronous function from asyncio, without blocking the main event loop, you can use loop.run_in_executor() to run the blocking function in a ThreadPoolExecutor or ProcessPoolExecutor` (i.e. in its own thread or process).

From within the async function you want to call it from:

loop = asyncio.get_event_loop()

result = await loop.run_in_executor(None, long_running_task, task)

The first argument None is to tell it to use the default executor for the loop. Obviously do_task() will still have to wait for result to complete, but while it is waiting, other async tasks will be able to run in event-loop.

edited Feb 14, 2019 at 9:51

answered Feb 14, 2019 at 7:37

J. Taylor

4,9255 gold badges40 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

y_159 Over a year ago

Is it possible to call multiple functions to run concurrently in the same loop.run_in_executor?

Leah Sapan Over a year ago

@y_159 check out asyncio.gather(), you can use it to run them in parallel.

y_159 Over a year ago

@LukeSapan asyncio.gather() is used to run tasks created by asyncio..create_task() concurrently. How would i use that here for loop.run_in_executor(). I think it starts executing the blocking function without creating a task object? My question is how to use a single loop.run_in_executor() to run multiple blocking functions concurrently.

Leah Sapan Over a year ago

@y_159 You can invoke loop.run_in_executor multiple times within asyncio.gather(...). Even though you're calling loop.run_in_executor multiple times, they'll all be using the default shared executor, so you aren't spawning new executors each time. Alternatively, if you don't want to use the default executor, you can create your own shared one, and pass it in as the first parameter to each call (instead of None as shown in the example above).

Collectives™ on Stack Overflow

calling sync functions from async function

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related