How to fetch status of javascript and css requests from network in selenium python

Question

I want to fetch the status of specific JavaScript and CSS network requests of my website https://lifesciences.cactusglobal.com. These network requests can be seen in chrome network. I want to print the request URL and its response code. Screenshot attached below for reference, check the underlined requests:

I tried using the package selenium-wire

driver.get('https://www.google.com')

for request in driver.requests:
    if request.response:
        print(
            request.url,
            request.response.status_code,
            request.response.headers['Content-Type']
        )

But it returns only responses and headers of the URL entered and not the network requests. Is there any other way to traverse through the network requests in selenium python?

Thanks

C. Peck · Accepted Answer · 2021-05-20 04:44:21Z

1

The output is a mess, but try getting the network requests like this, it seemed to work for me:

driver.get('https://lifesciences.cactusglobal.com/')
time.sleep(3)
test = driver.execute_script("var performance = window.performance || window.mozPerformance || window.msPerformance || window.webkitPerformance || {}; var network = performance.getEntries() || {}; return network;")

for item in test:
  print(item)

answered May 20, 2021 at 4:44

C. Peck

3,7193 gold badges21 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Libin Thomas Over a year ago

Many thanks @C. Peck. Your solution did work but I found another patch which suits more to my requirement. Adding those in the comments

Libin Thomas · Accepted Answer · 2021-05-21 06:19:44Z

1

Many thanks for your support. I found a solution which suits my requirement well.
Referred to these links:
https://gist.github.com/rengler33/f8b9d3f26a518c08a414f6f86109863c https://www.rkengler.com/how-to-capture-network-traffic-when-scraping-with-selenium-and-python/
Also found a package as an easy way out (not sticking to it though) - pip install selenium-wire

answered May 21, 2021 at 6:19

Libin Thomas

1,0941 gold badge11 silver badges28 bronze badges

Collectives™ on Stack Overflow

How to fetch status of javascript and css requests from network in selenium python

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related