Selenium Timeout Exception

Question

I am trying to scrape the cheapest flight price from Google Flight search page. I got a timeout error, even if I use WebDriverWait. This is my code:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

example_url = 'https://www.google.com/flights?hl=it#flt=/m/07_pf./m/01f62.2019-05-24;c:EUR;e:1;0px:2;sd:1;t:f;tt:o'

def get_price(url):
        driver = webdriver.Firefox()
        driver.get(url)
        try:
             wait = WebDriverWait(driver, 20)
             element = wait.until(EC.presence_of_element_located((By.CLASS_NAME, '.gws-flights-results__cheapest-price')))
             print(element)
        finally:
             price = driver.find_element_by_css_selector('.gws-flights-results__cheapest-price').text
             driver.quit()
        price = float(price.split(' ')[0])
        driver.quit()
        return price       


price = get_price(example_url)
print(price)

I got this error:

Traceback (most recent call last):
  File "semplice.py", line 23, in <module>
    price = get_price(example_url)
  File "semplice.py", line 13, in get_price
    element = wait.until(EC.presence_of_element_located((By.CLASS_NAME, '.gws-flights-results__cheapest-price')))
  File "/home/andrea/ENTER/lib/python3.4/site-packages/selenium/webdriver/support/wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

No message has been printed on the terminal. Just this is the message. Where is the problem ?

Marcel Wilson · Accepted Answer · 2019-05-22 04:39:52Z

2

@kafels is correct, you don't need the dot in front of a classname locator. However, once you fix that, you don't really need to do the 2nd search in the finally.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

example_url = 'https://www.google.com/flights?hl=it#flt=/m/07_pf./m/01f62.2019-05-24;c:EUR;e:1;0px:2;sd:1;t:f;tt:o'

def get_price(url):
    driver = webdriver.Firefox()
    driver.get(url)
    wait = WebDriverWait(driver, 20)
    try:
        element = wait.until(EC.presence_of_element_located((By.CLASS_NAME, 
                                                             'gws-flights-results__cheapest-price')))
    except Exception as exc:
        raise
    finally:
        # no need to look for the element here since you do it in the try section.
        driver.quit()

    print(element)
    price = element.text
    price = float(price.split(' ')[0])
    return price

price = get_price(example_url)
print(price)

answered May 22, 2019 at 4:39

Marcel Wilson

4,6284 gold badges35 silver badges67 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Andrea Barnabò Over a year ago

Thanks for your comment, however by using your suggestion I got a connection error "urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f778c7f8a58>: Failed to establish a new connection: [Errno 111] Connection refused " . However if I remove the dot, my code work. Thanks for your help

Marcel Wilson Over a year ago

I don't understand... there isn't a dot in my example.

Kafels · Accepted Answer · 2019-05-22 00:26:05Z

1

The error it is because you're passing the dot to find the element by class name:

element = wait.until(EC.presence_of_element_located((By.CLASS_NAME, '.gws-flights-results__cheapest-price'))

Change it to:

element = wait.until(EC.presence_of_element_located((By.CLASS_NAME, 'gws-flights-results__cheapest-price')))

answered May 22, 2019 at 0:26

Kafels

4,0891 gold badge18 silver badges33 bronze badges

Collectives™ on Stack Overflow

Selenium Timeout Exception

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related