Selenium (via Python) errors on SOME (but not all) :not(...) CSS selectors

Question

Running Python 2.7.8 & Selenium 3.11.0 (with Chrome webdriver), seeing some weird behavior. tl/dr, trying to use find_elements_by_css_selector, and it IS respecting a selector like p:not(.ignore) but NOT p:not(.ignore p) (though they both work in the real browser.

I'm serving the following webpage for test purposes:

<html>
    <title>Test</title>
    <body>
        <section class="ignore">
            <p>Some content I don't want to pull.</p>
        </section>
        <p>Content I DO want to pull.</p>
        <p>More important content.</p>
        <p>Thanks for reading.</p>
        <p class="also-ignore">(We CAN successfully ignore this one tho.)</p>
    </body>
</html>

And accessing it via Selenium with the following script:

#! /usr/bin/env python

from selenium import webdriver

if __name__ == '__main__':
    driver = webdriver.Chrome()

    try:
        driver.get('http://localhost:8000/test.html')
        elems = driver.find_elements_by_css_selector('p:not(.ignore p)')
        for e in elems:
            print e.text
    finally:
        driver.close()

The above throws the error:

Traceback (most recent call last):
  File "./test.py", line 10, in <module>
    elems = driver.find_elements_by_css_selector('p:not(.ignore p)')
  File "/Users/maiamccormick/code/seleniumtest/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 605, in find_elements_by_css_selector
    return self.find_elements(by=By.CSS_SELECTOR, value=css_selector)
  File "/Users/maiamccormick/code/seleniumtest/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 983, in find_elements
    'value': value})['value'] or []
  File "/Users/maiamccormick/code/seleniumtest/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
    self.error_handler.check_response(response)
  File "/Users/maiamccormick/code/seleniumtest/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified
  (Session info: chrome=65.0.3325.181)
  (Driver info: chromedriver=2.37.544337 (8c0344a12e552148c185f7d5117db1f28d6c9e85),platform=Mac OS X 10.13.4 x86_64)

(The selector p:not(.ignore p) DOES work in the Chrome console.)

The above Python code runs without error (and with expected output) with the CSS selector p:not(.also-ignore) (i.e. ignoring the last <p> element in the page), so the problem does not seem to be with the :not(...) selector itself. I'm stumped -- anyone have thoughts?

FWIW, replicated this behavior with the Firefox webdriver as well (same selectors do/don't work using Firefox webdriver -- and all work in Firefox console). — maiamcc
– maiamcc, Commented Apr 9, 2018 at 7:46
Apart from what is working and what not do you have a question for us? What is your exact usecase? — undetected Selenium
– undetected Selenium, Commented Apr 9, 2018 at 7:58
@DebanjanB tbh I'm not actually looking for help making something WORK -- I am easily able to hack around the problem above. I'm more curious WHY the selector didn't work as I expected it to. — maiamcc
– maiamcc, Commented Apr 12, 2018 at 17:17

Florent B. · Accepted Answer · 2018-04-09 15:05:23Z

3

From the W3C specification:

The negation pseudo-class, :not(X), is a functional notation taking a simple selector (excluding the negation pseudo-class itself) as an argument. It represents an element that is not represented by its argument.

This means that p:not(.ignore p) is not supported.

Now the reason it's working in the console is because the shorthand $ is overwritten by the page with JQuery which supports a different syntax.

Since Selenium rely on the CSS selector from the browser (same as document.querySelector) and not from JQuery, the exception in this case is to be expected.

Note that you'll get the exception in the console by either executing document.querySelector('p:not(.ignore p)') or $('p:not(.ignore p)') in a blank page (about:blank).

If you wish to use a JQuery selector, then use a JavaScript injection:

elms = driver.execute_script("return $('p:not(.ignore p)')")

edited Apr 9, 2018 at 15:05

answered Apr 9, 2018 at 14:51

Florent B.

42.7k7 gold badges92 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

BoltClock Over a year ago

See also: Why is my jQuery :not() selector not working in CSS?

Collectives™ on Stack Overflow

Selenium (via Python) errors on SOME (but not all) :not(...) CSS selectors

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related