Correctly implementing (concurrent) asyncio for multiple functions

Question

I have couple of functions and their execution is not dependent each other. What I am trying to do is execute them concurrently instead of sequentially (synchronous). I have added event loop as well, but I am not able to figure out if it is working correctly or not.

This is the implementation:

File 1:

import file2

def funcA():
    a, b = 1, 2
    file2.main(a, b)

File2:

def main(a, b):
    asyncio.get_event_loop().run_until_complete(_main(a, b))

async def _main(a, b):
     out1 = await funcA(a, b)
     out2 = await funcB(a, b)
     out3 = await funcC(a, b)

async def funcA(a, b):
    result = 1 # some processing done here
    return result

async def funcB(a, b):
    result = 1 # some processing done here
    return result

async def funcC(a, b):
    result = 1 # some processing done here
    return result

I am not able to figure out if these are working concurrently or not. I am adding time.sleep(10) in any function, executions stops there. I don't want them to run in background as I need output from those functions.

As a rule of thumb, async functions that don't await anything are most likely a wasted use of async. Unlike threads, asyncio is based on cooperative multitasking, and await (along with async for and async with) is the place where a context switch can happen. — user4815162342
– user4815162342, Commented Jun 4, 2020 at 11:06
good info. asyncio is still confusing for me even after trying it multiple times. I followed many tutorials but most of the them will show getting data from urls, instead of showing different examples. — Code Phat Gaya
– Code Phat Gaya, Commented Jun 4, 2020 at 14:28

mkrieger1 · Accepted Answer · 2023-08-31 08:13:36Z

10

One way to do what you want would be to use asyncio.run() in main and then gather in the async version of main. To simulate long processing, use asyncio.sleep() See the following code:

import asyncio

def main(a, b):
    res = asyncio.run(async_main(a, b))
    print(f"in main, result is {res}")

async def funcA(a, b):
    print('funcA - start')
    await asyncio.sleep(3)
    result = (a+b) # some processing done here
    print('funcA - end')

    return result

async def funcB(a, b):
    print('funcB - start')
    await asyncio.sleep(3)
    result = (a+b)*2 # some processing done here
    print('funcB - end')
    return result

async def funcC(a, b):
    print('funcC - start')
    await asyncio.sleep(3)
    result = (a+b)*3 # some processing done here
    print('funcC - end')

    return result

async def async_main(a, b):
    print("in async_main")
    res = await asyncio.gather(funcA(a, b), funcB(a, b), funcC(a, b))
    print(f"in async_main, result is {res}")
    return res

if __name__ == "__main__":
    main(1, 2)

The result is:

in async_main
funcA - start
funcB - start
funcC - start
funcA - end
funcB - end
funcC - end
in async_main, result is [3, 6, 9]
in main, result is [3, 6, 9]

edited Aug 31, 2023 at 8:13

mkrieger1

24.2k7 gold badges68 silver badges84 bronze badges

answered Jun 4, 2020 at 6:05

Roy2012

12.7k3 gold badges28 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Code Phat Gaya Over a year ago

one doubt, how am I supposed to get the output from funcA, funcB and funcC?

Code Phat Gaya Over a year ago

so in res = await asyncio.gather(funcA(a, b), funcB(a, b), funcC(a, b)), res will be a list or tuple, so that I can unpack it to other variables?

Roy2012 Over a year ago

yes. You get the result as a list, in which every element corresponds to one of the function calls - the first element to the first function you were 'gathering', the second to the second, etc. You can now do whatever you want with this list: unpack it, iterate over it, process it, etc.

Roy2012 Over a year ago

(BTW - if the post answers your question, do you mind 'accepting it' as an answer for future generations? )

Collectives™ on Stack Overflow

Correctly implementing (concurrent) asyncio for multiple functions

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related