PHP: How to scrape content of the website based on Javascript

Question

I'm trying to get content of this website using PHP simplehtmldom library.

http://www.immigration.govt.nz/migrant/stream/work/workingholiday/czechwhs.htm"

It is not working, so i tried using CURL:

function curl_get_file_contents($URL)
{
    $c = curl_init();
    curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($c, CURLOPT_URL, $URL);
    $contents = curl_exec($c);
    curl_close($c);

    if ($contents) return $contents;
    else return FALSE;
}

But always get only respose with some JS code and content:

<noscript>Please enable JavaScript to view the page content.</noscript>

Is any possibility to solve this using PHP? I must use PHP in this case so i need to simulate JS based browser.

Many thanks for any advice.

Igor Savinkin · Accepted Answer · 2020-01-02 15:04:01Z

2

I must use PHP in this case so i need to simulate JS based browser.

I'd recommend you two ways:

Leverage v8js php plugin to deal with site's js when scraping. See here an usage example.
Simulate JS based browser thru using Selenium, iMacros or webRobots.io Chrome ext. But in this case you are off the PHP scripting.

edited Jan 2, 2020 at 15:04

answered Mar 8, 2015 at 21:11

Igor Savinkin

6,35910 gold badges39 silver badges76 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Igor Savinkin Over a year ago

@redrom, thanks for checking my answer, could you share of what of those 2 options has helped you? And how did you apply it? I ask you this since i do web scraping research and then post results into the scraping.pro blog. Any feedback will be appreciable!

Collectives™ on Stack Overflow

PHP: How to scrape content of the website based on Javascript

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related