Why is the error "Cannot use HTMLSession within an existing event loop. Use AsyncHTMLSession instead"?

Question

I'm running the code provided by @Dan-Dev in his answer.

from requests_html import HTMLSession

url = 'https://www.thefreedictionary.com/love'
session = HTMLSession()
r = session.get(url)
r.html.render()
lang_bar = r.html.find('#LangBar', first=True)
print(lang_bar.html)

and the result is

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-ec1d9137b197> in <module>
      8 
      9 resp = session.get(url, headers = headers)
---> 10 resp.html.render()
     11 
     12 soup = bs(resp.html.html, "lxml")

C:\Anaconda3\lib\site-packages\requests_html.py in render(self, retries, script, wait, scrolldown, sleep, reload, timeout, keep_page)
    584         """
    585 
--> 586         self.browser = self.session.browser  # Automatically create a event loop and browser
    587         content = None
    588 

C:\Anaconda3\lib\site-packages\requests_html.py in browser(self)
    727             self.loop = asyncio.get_event_loop()
    728             if self.loop.is_running():
--> 729                 raise RuntimeError("Cannot use HTMLSession within an existing event loop. Use AsyncHTMLSession instead.")
    730             self._browser = self.loop.run_until_complete(super().browser)
    731         return self._browser

RuntimeError: Cannot use HTMLSession within an existing event loop. Use AsyncHTMLSession instead.

Clearly, the code runs fine in Dan-Dev's computer. Could you please explain why my laptop returns an error Cannot use HTMLSession within an existing event loop. Use AsyncHTMLSession instead?

Can you post details of your environment OS Python version etc? Are you using Jupyter Notebook? Have you tried using AsyncHTMLSession instead of HTMLSession? — Dan-Dev
– Dan-Dev, Commented Aug 23, 2020 at 16:29

blondelg · Accepted Answer · 2021-06-11 11:47:44Z

5

You may try this:

import nest_asyncio

nest_asyncio.apply()

session = HTMLSession()
r = session.get("URL")

html_str = r.text

edited Jun 11, 2021 at 11:47

answered Sep 23, 2020 at 7:20

blondelg

9961 gold badge9 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Sean McCarthy Over a year ago

Where does url_template come from? No import in your original answer.

blondelg Over a year ago

I was usiing a url template to build url dynamically. This is not important here so I have just replaced this by a random url.

nphaibk Jan 6 at 20:17

The error comes from r.html.render(), you omitted that line in your code

Carlost · Accepted Answer · 2022-10-01 17:47:37Z

5

Use arender() instead of render() or set the script as async

answered Oct 1, 2022 at 17:47

Carlost

90511 silver badges18 bronze badges

Comments

Ian · Accepted Answer · 2022-04-28 15:38:39Z

2

You can save these codes in a source file, such as 'src.py'. Then open an Anaconda Prompt (assuming you are using this) and run this command (assure you are in the folder containing the 'src.py' file)

python src.py

This solution has worked for me. You should not run the source file in Spyder IDE or Jupyter Notebook.

answered Apr 28, 2022 at 15:38

Ian

1701 silver badge12 bronze badges

1 Comment

nphaibk Jan 6 at 20:07

I confirm this works, but not so convenient (you need to activate the correct env, and have a separate file)

Trop · Accepted Answer · 2024-03-03 19:42:23Z

1

This code may works because it needs some periods time to render the object.

r.html.arender(timeout=0,sleep=20)

edited Mar 3, 2024 at 19:42

Trop

24 bronze badges

answered Jul 1, 2023 at 10:10

kam kam

112 bronze badges

1 Comment

ryanwebjackson Over a year ago

Please add a citation for this.

Collectives™ on Stack Overflow

Why is the error "Cannot use HTMLSession within an existing event loop. Use AsyncHTMLSession instead"?

4 Answers 4

3 Comments

Comments

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related