Python-Selenium: Chrome headless setting does not work with "WebDriverWait"

Question

The following code passes with real browser but no with headless chrome.

 while True:
    try:
        #Explicitly wait
        print("Try4")
        WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "00BC0000008hfCI_listSelect")))
        print("Try5")
        ##Locate the select box
        s = driver.find_element_by_id("00BC0000008hfCI_listSelect")
        ##Unassigned queue
        Select(s).select_by_value("00BC0000008hqG8")
        time.sleep(1)
    except TimeoutException :
        print("Wrong!")
        break

The result will be:

Try4
Wrong!

It seems that the headless chrome can't find the element. It wowrks well with Chrome browser. My Selenium version is 3.14.0.

The HTML will be like:

<div class="topNav primaryPalette" id="00BC0000008hfCI_topNav"><div class="controls"><img src="/img/s.gif" alt="Case" class="pageTitleIcon" title="Case">
<select class="title" id="00BC0000008hfCI_listSelect" name="fcf" onchange="ListViewport.instances['00BC0000008hfCI'].showFeedOrListView(this.options[this.selectedIndex].value)" title="View:"><option value="00BC0000009AFei">All Open Casa Cases</option>
    <option value="00B800000063tb2">All Open Cases</option>
    <option value="00B1A000009ZCok">Channel Upgrade Cases CCLTD</option>
    <option value="00B1A000009VKSg">Channel Upgrade Queue</option>
    <option value="00BC00000099n8S">China team's cases</option>
    <option value="00BC00000097nXg">Delete Queue</option>
    <option value="00BC00000097FPw">Handoff Case Queue</option>
    <option value="00B0y000009vcEI">Maintenance Windows Due Today</option>
    <option value="00B1A000009LY1L">mark r upgrade</option>
    <option value="00B1A000009V2x7">My close case</option>
    <option value="00BC000000976jF">My Open Cases</option>
    <option value="00B0y000009g0J9">My Open Cases - Final Solution Provided</option>
    <option value="00BC0000008hfCI" selected="selected">New Cases Created Today</option>
    <option value="00B0y000009vWOH">New Cases Created Today All</option>
    <option value="00B800000063tb0">Recently Viewed Cases</option>
    <option value="00BC00000099gHm">Stale Case View</option>
    <option value="00BC0000008hqG8">Unassigned Case Queue</option>
    </select><div class="filterLinks" id="00BC0000008hfCI_filterLinks"><a href="/ui/list/FilterEditPage?id=00BC0000008hfCI&amp;retURL=%2F500%3Ffcf%3D00BC0000008hfCI%26rolodexIndex%3D-1%26page%3D1">Clone</a> | <a id="create-new-view" href="/ui/list/FilterEditPage?ftype=t&amp;retURL=%2F500%3Ffcf%3D00BC0000008hfCI%26rolodexIndex%3D-1%26page%3D1">Create New View</a></div><span class="divisionLabel" id="00BC0000008hfCI_divisionLabel"></span><div class="topNavTab" id="00BC0000008hfCI_topNavTab"><div class="toggleButton" id="00BC0000008hfCI_toggleButton"><span class="listToggle chatterListToggle selected"><a href="javascript:void(0);" class="toggleLink" onclick="ListViewport.instances['00BC0000008hfCI'].toggleFeedListView(false)" title="View List - Selected"><span class="toggleIcon listIcon"></span><span class="linkText">List</span></a></span><span class="chatterListToggle chatterToggle clickable" id="ext-gen3"><a href="javascript:void(0);" class="toggleLink" onclick="ListViewport.instances['00BC0000008hfCI'].toggleFeedListView(true)" title="View Feed"><span class="toggleIcon chatterIcon"></span><span class="linkText">Feed</span></a></span></div><img src="/img/s.gif" alt="" class="printerIconDisabled" id="00BC0000008hfCI_printLinkDisabled" style="display: none" title="Printable view is not available for feeds."><a href="javascript:printWin('/500/x?fcf=00BC0000008hfCI&amp;rolodexIndex=-1&amp;page=1')" id="00BC0000008hfCI_printLink" style="" title="Printable View (New Window)"><img src="/img/s.gif" alt="Printable View (New Window)" class="printerIcon" onblur="this.className = 'printerIcon';" onfocus="this.className = 'printerIconOn';" onmouseout="this.className = 'printerIcon';this.className = 'printerIcon';" onmouseover="this.className = 'printerIconOn';this.className = 'printerIconOn';" title="Printable View (New Window)"></a><a href="javascript:openPopupFocusEscapePounds(%27https://login.salesforce.com/services/auth/sso/00D30000000XsfGEAS/HTAuthProvider?startURL=%252Fapex%252Fhtdoor%253Floc%253Dhelp%2526target%253Dcases_view.htm%2526section%253DCases%2526language%253Den_US%2526release%253D214.17.3%2526instance%253DNA66&amp;site=https%3A%2F%2Fhelp.salesforce.com&amp;showSplash=true%27, %27Help%27, 1024, 768, %27width=1024,height=768,resizable=yes,toolbar=yes,status=yes,scrollbars=yes,menubar=yes,directories=no,location=yes,dependant=no%27, false, false);" title="Help for this Page (New Window)"><img src="/img/s.gif" alt="Help for this Page (New Window)" class="helpIcon" title="Help for this Page (New Window)"></a></div><div class="clearingBox"></div></div></div>

I tried the two ways but no help. chrome_options.add_argument('window-size=1920,1080') and capabilities = DesiredCapabilities.CHROME.copy()

Would you please help? Thanks in advance.

How can Selenium rather WebDriver in conjunction with WebDriverWait locate the presence of element when the element is not present at all? — undetected Selenium
– undetected Selenium, Commented Aug 17, 2018 at 9:04
@DebanjanB It is part of the code. It should always exists in the page, else it will have timeoutexception and break the loop. I have imported webdriver. — Winnie-c
– Winnie-c, Commented Aug 17, 2018 at 10:42
But I suppose your locator strategy is a bit off. Can you update the question with the relevant HTML? — undetected Selenium
– undetected Selenium, Commented Aug 17, 2018 at 10:44
@DebanjanB I updated my question. The code actually can work without "headless" setting. I think there may be something related to the headless character.. — Winnie-c
– Winnie-c, Commented Aug 17, 2018 at 11:34

josephnvu · Accepted Answer · 2024-02-24 12:31:52Z

4

Adding a real browser user-agent header works for me:

user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36'

chrome_options.add_argument('user-agent={0}'.format(user_agent))

edited Feb 24, 2024 at 12:31

josephnvu

1,2521 gold badge9 silver badges14 bronze badges

answered Jan 5, 2023 at 0:47

Gabriel Nunes

413 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

undetected Selenium · Accepted Answer · 2018-08-17 12:37:25Z

0

As per the HTML you have shared and your code trials instead of using the expected-conditions method presence_of_element_located() you need to use the method visibility_of_element_located() and you can use either of the following solutions:

CSS_SELECTOR:

WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "select.title[name='fcf'][id$='_listSelect']")))

XPATH:

WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//select[@class='title' and @name='fcf'][contains(@id,'_listSelect')]")))

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

answered Aug 17, 2018 at 12:37

undetected Selenium

194k44 gold badges304 silver badges387 bronze badges

1 Comment

Winnie-c Over a year ago

Hello, thanks for checking the problem. I still get the same error like before. It goes well with the real browser.

Marc · Accepted Answer · 2020-03-09 22:55:31Z

0

How to go headless in Py Selenium using WebDriverWait for anyone looking for help.

from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")

driver = webdriver.Chrome(options=chrome_options). 
WebDriverWait(driver, ...)

The options keyword used to be chrome_options.

answered Mar 9, 2020 at 22:55

Marc

1,99719 silver badges26 bronze badges

Collectives™ on Stack Overflow

Python-Selenium: Chrome headless setting does not work with "WebDriverWait"

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related