No Visible Text Results from Python XPATH Selenium Elements

Question

I would like to extract the "First Published" dates using selenium on Python, yet I am facing problems trying to get any visible date results even though I'm getting successful results when looking up the xpath through the browser's inspection tab, I do get successful length results of the elements, yet no text results of the dates on my console.

My Code:

import time

from selenium import webdriver
driver = webdriver.Firefox(executable_path=r'C:\\Users\\MyComputer\\PycharmProjects\\SeleniumProject\\venv\\Lib\site-packages\\selenium\\webdriver\\common\\geckodriver.exe')
driver.get('https://tools.cisco.com/security/center/publicationListing.x?resourceIDs=93036,5834,80720&apply=1,1,1&totalbox=3&pt0=Cisco&cp0=93036&pt1=Cisco&cp1=5834&pt2=Cisco&cp2=80720#~FilterByProduct')

time.sleep(20)

prices = driver.find_elements_by_xpath('//span[@class="ng-binding" and contains(text(),"GMT")]')

for post in prices:
    if post.text != "":
        print(post.text)

print(len(prices))

driver.close()

I have tried other visible xpaths on the website to test on python and I can get the 20 vulnerability titles that show up on screen as seen when you open the link, so I am assuming that I have to tell selenium to click every link and extract the date and do that for every title ? But then how am I able to get them all in one go through the browser inspection tab ?

All help is appreciated, Thanks,

Guy · Accepted Answer · 2019-11-17 05:45:35Z

1

Selenium can retrieve only visible text. If you don't won't to open all the hidden sections you can use get_attribute('textContent') or get_attribute('innerText')

for post in prices:
    print(post.get_attribute('innerText'))

answered Nov 17, 2019 at 5:45

Guy

51.2k10 gold badges49 silver badges96 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Zeki Ramahi Over a year ago

I have tried that option before but still had no results, @Rock's Code worked perfectly, Thanks though.

Zeki Ramahi Over a year ago

Retried the command on ChromeDriver and worked like a charm, the previous Suggested answer showed the visible date instead of the non-visible one which I wanted. Many thanks for your suggestion sir.

SeleniumUser · Accepted Answer · 2019-11-19 21:31:40Z

0

Please find a solution if you want to fetch first publish date

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium import webdriver



driver = webdriver.Chrome(executable_path=r"C:\New folder\chromedriver.exe")
driver.maximize_window()

driver.get("https://tools.cisco.com/security/center/publicationListing.x?resourceIDs=93036,5834,80720&apply=1,1,1&totalbox=3&pt0=Cisco&cp0=93036&pt1=Cisco&cp1=5834&pt2=Cisco&cp2=80720#~FilterByProduct")
print driver.current_url

element = WebDriverWait(driver, 15).until(EC.visibility_of_element_located((By.XPATH, "//div[@id='tab-2']//tr[1]//td[1]//table[1]//tbody[1]//tr[1]/td[4]/span")))
print element.text

If you want to print all dates

dates = WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.XPATH, "//div[@id='tab-2']//tr[*]//td[1]//table[1]//tbody[1]//tr[*]/td[4]/span")))
print(len(dates))


for date in dates:
    print date.text

edited Nov 19, 2019 at 21:31

answered Nov 17, 2019 at 0:48

SeleniumUser

4,1973 gold badges15 silver badges45 bronze badges

4 Comments

Zeki Ramahi Over a year ago

Thank you so much for your time sir, the performance and results now work very smooth thanks to you!

SeleniumUser Over a year ago

@ZekiRamahi: Sorry mate but based on your previous comment I thought your issue was resolved

Zeki Ramahi Over a year ago

Sorry for removing your comment as the answer but your code showed me the "visible" date which is the "update date" I wanted the "published date" which was stored inside every vulnerability which can be seen when you click on them. However, the performance upgrade was a significant boost on the runtime of the code. Thank you nonetheless.

SeleniumUser Over a year ago

Cool but can I request you if your issue is not resolved then please mention that and don't accept answer going forward ;)

Collectives™ on Stack Overflow

No Visible Text Results from Python XPATH Selenium Elements

2 Answers 2

2 Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related