How do I run Python asyncio code in a Jupyter notebook?

Question

I have some asyncio code which runs fine in the Python interpreter (CPython 3.6.2). I would now like to run this inside a Jupyter notebook with an IPython kernel.

I can run it with

import asyncio
asyncio.get_event_loop().run_forever()

and while that seems to work it also seems to block the notebook and doesn't seem to play nice with the notebook.

My understanding is that Jupyter uses Tornado under the hood so I tried to install a Tornado event loop as recommended in the Tornado docs:

from tornado.platform.asyncio import AsyncIOMainLoop
AsyncIOMainLoop().install()

However that gives the following error:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-1-1139449343fc> in <module>()
      1 from tornado.platform.asyncio import AsyncIOMainLoop
----> 2 AsyncIOMainLoop().install()

~\AppData\Local\Continuum\Anaconda3\envs\numismatic\lib\site- packages\tornado\ioloop.py in install(self)
    179         `IOLoop` (e.g.,     :class:`tornado.httpclient.AsyncHTTPClient`).
    180         """
--> 181         assert not IOLoop.initialized()
    182         IOLoop._instance = self
    183 

AssertionError:

Finally I found the following page: http://ipywidgets.readthedocs.io/en/stable/examples/Widget%20Asynchronous.html

so I added a cell with the following code:

import asyncio
from ipykernel.eventloops import register_integration

@register_integration('asyncio')
def loop_asyncio(kernel):
    '''Start a kernel with asyncio event loop support.'''
    loop = asyncio.get_event_loop()

    def kernel_handler():
        loop.call_soon(kernel.do_one_iteration)
        loop.call_later(kernel._poll_interval, kernel_handler)

    loop.call_soon(kernel_handler)
    try:
        if not loop.is_running():
            loop.run_forever()
    finally:
        loop.run_until_complete(loop.shutdown_asyncgens())
        loop.close()

and in the next cell I ran:

%gui asyncio

That worked but I don't really understand why and how it works. Can someone please explain that to me?

felipe · Accepted Answer · 2024-08-03 15:36:34Z

99

EDIT FEB 21st, 2019: Problem Fixed

This is no longer an issue on the latest version of Jupyter Notebook. Authors of Jupyter Notebook detailed the case here.

You may now just do:

import asyncio

async def func():
    print("started.")
    await asyncio.sleep(2)
    print("done.")

await func()

Answer below was the original response that was marked correct by the op.

This was posted quite a bit ago, but in case other people are looking for an explanation and solution to the problem of running asynchronous code inside Jupyter Notebook;

Jupyter's Tornado 5.0 update bricked asyncio functionalities after the addition of its own asyncio event loop:

Thus, for any asyncio functionality to run on Jupyter Notebook you cannot invoke a loop.run_until_complete(...), since the loop you will receive from asyncio.get_event_loop() will be active.

Instead, you must either add the task to the current event loop:

import asyncio
loop = asyncio.get_event_loop()
loop.create_task(some_async_function())

Or get the results via run_coroutine_threadsafe:

import asyncio
loop = asyncio.get_event_loop()
asyncio.run_coroutine_threadsafe(some_async_function(), loop)

edited Aug 3, 2024 at 15:36

answered Jul 14, 2018 at 18:49

felipe

8,1253 gold badges30 silver badges40 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Gary Li Over a year ago

Is there any document for this behavior?

QHarr Over a year ago

Really helpful thanks. + Does anything need to be closed in the above or is it complete as is?

felipe Over a year ago

@QHarr If you close the loop, then Jupyter stops working.

Giorgio Balestrieri Over a year ago

notice this answer is not anymore up to date. this is

felipe Over a year ago

@GiorgioBalestrieri Updated post.

James D · Accepted Answer · 2018-12-02 15:30:55Z

90

This is no longer an issue in the latest jupyter release!

https://blog.jupyter.org/ipython-7-0-async-repl-a35ce050f7f7

Just write an async function, and then await it directly in a jupyter cell.

async def fn():
  print('hello')
  await asyncio.sleep(1)
  print('world')

await fn()

answered Dec 2, 2018 at 15:30

James D

1,7941 gold badge15 silver badges10 bronze badges

3 Comments

Joël Over a year ago

Thanks for providing the command to apply, I was kinda stuck using the old workaround :)

curiouz Over a year ago

From all others solution, this is the most elegant one!

Tanner Davis Over a year ago

Thanks for linking the reference to the fix as well! I thought I was going to lose my mind using the event loop and thread safe options.

Дмитро Олександрович · Accepted Answer · 2019-05-21 14:41:16Z

My aha moment with Asyncio in Jupyter looks like this:

import time,asyncio

async def count():
    print("count one")
    await asyncio.sleep(1)
    print("count four")

async def count_further():
    print("count two")
    await asyncio.sleep(1)
    print("count five")

async def count_even_further():
    print("count three")
    await asyncio.sleep(1)
    print("count six")

async def main():
    await asyncio.gather(count(), count_further(), count_even_further())

s = time.perf_counter()
await main()
elapsed = time.perf_counter() - s
print(f"Script executed in {elapsed:0.2f} seconds.")

Output:

count one
count two
count three
count four
count five
count six
Script executed in 1.00 seconds.

Originally from here, but the example was not clear for me at first: https://realpython.com/async-io-python/

kiranr · Accepted Answer · 2022-08-29 10:54:06Z

13

just use %autoawait magic command,

eg :

%autoawait asyncio

await some_function()

answered Aug 29, 2022 at 10:54

kiranr

2,4831 gold badge18 silver badges36 bronze badges

2 Comments

Jeroen Vermunt Over a year ago

Perfect! This worked for me, if it does for others this should be the right answer.

DUMBA Over a year ago

this did things for me

dmmfll · Accepted Answer · 2018-07-20 11:03:13Z

3

I recently ran into the issue of not being able to run asyncio code in a Jupyter notebook. The issue is discussed here: https://github.com/jupyter/notebook/issues/3397

I tried one of the solutions in the discussion and it solved the issue so far.

pip3 install tornado==4.5.3

This replaced tornado version 5.x that was installed by default.

The asyncio code in a Jupyter notebook then ran as expected.

answered Jul 20, 2018 at 11:03

dmmfll

2,8642 gold badges38 silver badges43 bronze badges

4 Comments

igor Over a year ago

Were you able to rerun jupyter cell with async code?

dmmfll Over a year ago

Yes. So far all is working as expected. I had to comment out some loop.close().

igor Over a year ago

So, only removing loop.close guarantees rerunability?

dmmfll Over a year ago

I am unqualified say if it guarantees rerunability. It functioned as I expected in the particular context in which I was working. RuntimeErrors went away after downgrading to tornado 4.5.3.

Collectives™ on Stack Overflow

How do I run Python asyncio code in a Jupyter notebook?

5 Answers 5

5 Comments

3 Comments

Comments

2 Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

5 Comments

3 Comments

Comments

2 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related