Understanding asyncio in python

Question

I've been having a hard time understanding Python's asyncio module and how to not block on asynchronous calls. For instance, given this code snippet:

import aiohttp
import asyncio
import async_timeout

async def fetch(session, url):
    with async_timeout.timeout(10):
        async with session.get(url) as response:
            return await response.text()

async def main(loop):
    print(1)
    async with aiohttp.ClientSession(loop=loop) as session:
        html = await fetch(session, 'http://python.org')
        print(html)
    print(2)

loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))

I would expect, similar to how this would work in Javascript, the output to be

1
2
<!doctype html>
<...>

Instead, the function prints 1, blocks until it gets back the html, then prints 2. Why does it block, and how / can I avoid the blocking? Thanks for your help.

Community · Accepted Answer · 2017-05-23 12:09:44Z

3

Your problem is this line:

html = await fetch(session, 'http://python.org')

this means - wait until getting a response from fetch, Or in other word's it's blocking the current coroutine until the fetch coroutine return's the response text.

To make things simpler:

The fetch operation by itself isn't blocking, only the await.
The await is blocking only for the current coroutine, not the thread.
Like vincent said, calling the operation by itself doesn't do anything (we don't start the fetching process)

from asyncio documentation

Calling a coroutine does not start its code running – the coroutine object returned by the call doesn’t do anything until you schedule its execution. There are two basic ways to start it running: call await coroutine or yield from coroutine from another coroutine (assuming the other coroutine is already running!), or schedule its execution using the ensure_future() function or the AbstractEventLoop.create_task() method.

And a fix (for getting the desired output):

import asyncio
import aiohttp
import async_timeout


async def fetch(session, url):
    with async_timeout.timeout(10):
        async with session.get(url) as response:
            return await response.text()


async def main(loop):
    print(1)
    async with aiohttp.ClientSession(loop=loop) as session:
        future = asyncio.ensure_future(fetch(session, 'http://python.org')) # we start the fetch process
        print(2)
        html = await future # we wait for getting the fetch response
        print(html)


loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))

edited May 23, 2017 at 12:09

CommunityBot

11 silver badge

answered Feb 23, 2017 at 7:33

etlsh

7011 gold badge8 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Vincent Over a year ago

Calling the fetch coroutine doesn't schedule the fetch operation, and it doesn't return a future. You have to use asyncio.ensure_future in order to achieve that.

etlsh Over a year ago

Thanks for correcting my answer, I edited my code and added additional explanation.

rumdrums Over a year ago

Thanks, this explanation was very helpful. So, if I did want to start the coroutine without blocking execution of the rest of the function... I would effectively need to await an asyncio.sleep(0) after the asyncio.ensure_future?? Is there a better way to do that?

etlsh Over a year ago

using asyncio.ensure_future start the coroutine and dosn't block the rest of the function

rumdrums Over a year ago

@etlsh I guess what I mean is that, if I add an await asyncio.sleep(0) just after the asyncio.ensure_future, it will actually start executing the coroutine before 2 prints. Otherwise, it won't even begin until after the 2 prints, correct?

|

dekassert · Accepted Answer · 2017-02-23 21:48:44Z

3

To understanding asyncio in python 3.6 async def style I suggest you to run this simple example:

import asyncio
import time

def write(msg):
    print(msg, flush=True)

async def say1():
    await asyncio.sleep(1)
    write("Hello 1!")

async def say2():
    await asyncio.sleep(1)
    write("Hello 2!")

write("start")
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(
    say1(),
    say2()
))
write("exit")

loop.close()

Source: https://maketips.net/tip/146/parallel-execution-of-asyncio-functions See link for detailed explanations of this code with call diagram .

answered Feb 23, 2017 at 21:48

dekassert

313 bronze badges

Collectives™ on Stack Overflow

Understanding asyncio in python

2 Answers 2

6 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related