2

I need to create a Runner class that can asynchronously runs multiple jobs and return the results one all jobs are completed. Here is the dummy code snippet which I already have

import asyncio

class Runner:
  def __init__(self):
    self.jobs_loop = asyncio.new_event_loop()

  async def slow_process(self, *args):
    ...  # run slow proces with args
    await asyncio.sleep(5)
    return "result"

  def batch_run(self, count, *args):
    results = self.jobs_loop.run_until_complete(asyncio.gather(
      *[self.slow_process(*args) for _ in range(count)],
      loop=self.jobs_loop,
      return_exceptions=True
    ))
    return results

runner = Runner()
print(runner.batch_run(10))

When the batch_run is called, it schedules count jobs and blocks until finished. This, however, doesn't work when called from a running event loop raising RuntimeError: Cannot run the event loop while another loop is running

How can I make batch_run work when called from existing event loop (e.g. Jupyter Console) as well as being callable from outside of the event loop.

EDIT

Ideally the function would look like this

def batch_run(self, count, *args):
  loop = asyncio.get_event_loop()
  jobs = asyncio.gather(...)
  if not loop.is_running():
    return loop.run_until_complete(jobs)
  else:
    future = asyncio.ensure_future(jobs, loop=loop)
    # loop.wait_until_completed(future)
    return future.result()

if only loop.wait_until_completed existed.

3
  • Possibly duplicate of stackoverflow.com/questions/47518874/… Commented Jul 28, 2019 at 14:44
  • 1
    Not really a duplicate. This problem is not strictly related to the Jupyter console but is more generic question about how to make a function callable from both within and without asyncio loop. Commented Jul 28, 2019 at 14:54
  • What about passing the loop at the class constructor? There is a reason that batch_run is not async as well? Commented Dec 30, 2019 at 12:23

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.