Scraping HTML page using XPath and PHP

Question

I'm trying to scraping a HTML page using this PHP code

<?php
    ini_set('display_errors', 1);

    $url = 'http://www.cittadellasalute.to.it/index.php?option=com_content&view=article&id=6786:situazione-pazienti-in-pronto-soccorso&catid=165:pronto-soccorso&Itemid=372';


    //#Set CURL parameters: pay attention to the PROXY config !!!!
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
    curl_setopt($ch, CURLOPT_PROXY, '');
    $data = curl_exec($ch);
    curl_close($ch);

    $dom = new DOMDocument();
    @$dom->loadHTML($data);

    $xpath = new DOMXPath($dom);

    $greenWaitingNumber = $xpath->query('/html/body/div/div/div[4]/div[3]/section/p');


    foreach( $greenWaitingNumber as $node )
    {
      echo "Number first green line: " .$node->nodeValue;
      echo '<br>';
      echo '<br>';
    }


?>

All works fine (no error and in my browser console I can see '200' as return code ...), but nothing is printed in my HTML page .... .

Probably the problem is about the xpath /html/body/div/div/div[4]/div[3]/section/p that refers to the first green line in the source HTML page, but this is my Firefox Firebug tells me for that page section ....

Suggestions / examples?

!!! UPDATE !!!!

As Santosh Sapkota suggest in his reply, the first problem is that the text inside that green box, is loaded from iFrame ... I've seen the url of the HTML page inside the IFrame ad so I've tried to use this one in my code that now is ...

<?php
    ini_set('display_errors', 1);

    $url = 'http://listeps.cittadellasalute.to.it/?id=01090101';


    //#Set CURL parameters: pay attention to the PROXY config !!!!
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
    curl_setopt($ch, CURLOPT_PROXY, '');
    $data = curl_exec($ch);
    curl_close($ch);

    $dom = new DOMDocument();
    @$dom->loadHTML($data);

    $xpath = new DOMXPath($dom);

    $greenWaitingNumber = $xpath->query('/html/body/div/div/div[4]/div[3]/section/p');


    foreach( $greenWaitingNumber as $node )
    {
      echo "Number first green line: " .$node->nodeValue;
      echo '<br>';
      echo '<br>';
    }


?>

but unfortunately nothing is still printed in my output HTML page ....

Other suggestions / examples?

Santosh Sapkota · Accepted Answer · 2017-08-11 14:28:47Z

1

Must be problem with you xpath. As well as check if there is content laded from iFrame or not.

answered Aug 11, 2017 at 14:28

Santosh Sapkota

1405 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Cesare Over a year ago

How may I get the right xpath when there is an Iframe?

Santosh Sapkota Over a year ago

if you are trying to get text inside that green box, you can clearly see that that is loaded from iFrame.

Collectives™ on Stack Overflow

Scraping HTML page using XPath and PHP

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related