1

Currently, asyncio Event Loop supports non-blocking work with network sockets, files, subprocesses, and signals. For other operations, documentations describes executing code in thread or process pools, but that seems much less efficient (threads vs coroutines).

Is there a way to introduce additional low-level non-blocking operations, e.g. wrapping I/O-heavy library calls? What primitives must be implemented?

4
  • Can you give an example of the kind of low-level call you'd like to introduce? Is the underlying call asynchronous itself? If so, does it provide a notification mechanism or is the caller expected to poll it for completion? Commented Oct 28, 2018 at 20:10
  • No, the underlying library call is blocking. I'm thinking of the non-blocking interface extension to introduce, so it's compatible with asyncio. For the sake of definiteness, let's consider Windows Overlapped Serial port communication as example of non-blocking interface. Commented Oct 29, 2018 at 7:36
  • 1
    If the underlying library is blocking and you want to wrap it (rather than reimplement it), then threads might be your only option. Commented Oct 29, 2018 at 8:33
  • I have control over the library code, so I can extend the interface with non-blocking calls. But I totally miss what calls are expected by asyncio. Commented Oct 29, 2018 at 8:53

1 Answer 1

1

Like most event loops, the asyncio event loop is built around polling IO sources, file descriptors on Unix and file handles on Windows. Poll or select is an IO operation that efficiently monitors multiple file descriptors, suspending the current thread until something interesting happens, e.g. new data arrives. Optionally, polling accepts a timeout, so execution can continue even if there is no new IO event. For a more detailed treatment of the topic, refer to this SO answer and other answers to the same question.

The poll-based design allows asyncio to natively support two kinds of events:

  • IO possible
  • timeout expired

All other types of events must be expressed in terms of those two. For example, call_soon_threadsafe wakes up the event loop by writing into an internal pipe monitored by the event loop. The implementation of run_in_executor uses call_soon_threadsafe to inform asyncio that the synchronous function has completed.

To connect an entirely different polling mechanism to asyncio, one would typically spawn a thread dedicated to that kind of polling, and use call_soon_threadsafe to inform asyncio of new events. This answer, for example, shows how to connect the raw multiprocessing pool (which can be terminated) to asyncio.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.