PHP web based scraper

Question

What Im trying to do is use PHP to scrape a website of a url I enter into a parameter.

I want the whole raw source code.. But thats not all..

I want it then saved into an html page, and onto the local server of the php script.

Is there a Easy Snippet for this? or can someone easily write me up a code?

For example

So for instance, mysite.com/scrape.php?url=http://google.com

I want it to save the front page of google into http://mysite.com/scraped/google.com.html

How about looking into the documentation for file_get_contents() and file_put_contents() — Cyclonecode
– Cyclonecode, Commented Nov 24, 2012 at 6:26

Cyclonecode · Accepted Answer · 2016-07-18 11:49:16Z

2

Here's a script that will save the contents of the specified url into a file named scraped.html:

if (isset($_GET['url'])):
   $contents = file_get_contents($_GET['url']);
   file_put_contents('scraped.html', $contents);
endif;

To use a url in the call to file_get_contents() you must enable allow_url_fopen in your php.ini file.

Of course this will only save the actual source of the requested url and not any other resources, such as images, scripts and stylesheets.

answered Nov 24, 2012 at 6:29

Cyclonecode

30.4k11 gold badges77 silver badges97 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1