Scrape a website (javascript website) using php

Question

I am trying to scrape a website (believe it is in JavaScript) using a simple PHP script. I am a beginner so any help would be greatly appreciated. The URL of the webpage is:

http://www.indiainfoline.com/Markets/Company/Fundamentals/Balance-Sheet/Yes-Bank-Ltd/532648

So here for example I would like to pass the name of company (Yes-Bank-Ltd) and code (532648) in get_file_contents. Not sure on how to do it so can somebody please help.

Thanks, Nidhi

what do you mean by pass it, as I can see they are already in url — Senad Meškin
– Senad Meškin, Commented Jul 11, 2011 at 18:15
@Nightfirecat: Ahh, makes a lot more sense! Though I would have loved the idea of a website scrapyard, with lots of old queries and dying animated GIFs... — Kerrek SB
– Kerrek SB, Commented Jul 11, 2011 at 18:17

Patrick Desjardins · Accepted Answer · 2011-07-11 18:16:49Z

1

Why aren't you just not append the string of the company and code in the url. Here is an idea that you fill up an array of company and code (need to be the same size) and then you loop them to scrape the data you want.

for($i=0;$i<count($listOfCie);$i++)
{
    $cie = $listOfCie[$i];
    $code = $listOfCode[$i];
    $urlToScrape = "http://www.indiainfoline.com/Markets/Company/Fundamentals/Balance-Sheet/" . $cie . "/" . $code
    //... = get_file_contents($urlToScrape....
 }

answered Jul 11, 2011 at 18:16

Patrick Desjardins

142k89 gold badges298 silver badges346 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user839403 Over a year ago

Ok, seems to be a good idea. However, when i try using hte syntax you gave here, it is giving a problem. Can you please verify if you need two quotes in the first variable and no quotes after the second one.

Patrick Desjardins Over a year ago

Are you a developer? Quotes verifications should be your job to check. It's just a pseudo code to show you how to solve your problem.

user839403 Over a year ago

no, i am not a developer, and hence the issue.. sorry for causing you trouble

Patrick Desjardins Over a year ago

Stackoverflow is not a place to get code all made for you project but a place to ask question. Sorry I can't give you more than that, I evaluate my answer as a good start. You should get some PHP tutorial and come back here because string concatenation is basic stuff.

user839403 Over a year ago

Ok, i figured it out. I was using the command inside a single quote and not a double quote. Now this is working. BTW, would you know how i can run a javascript on the page that i am scrapping. In this case, i want consolidated results and hence javasript:_doPostBack()

TimoSolo · Accepted Answer · 2018-01-17 11:11:17Z

0

The simplest way to scrape a site in PHP is to use curl (http://php.net/manual/en/book.curl.php)

For some examples look at http://php.net/manual/en/curl.examples-basic.php or google :)

If the website relies on javascript though it's going to be difficult to get the data you want. You might look at a "headless browser" like http://phantomjs.org/

answered Jan 17, 2018 at 11:11

TimoSolo

7,3656 gold badges36 silver badges51 bronze badges

Comments

Tilo Mitra · Accepted Answer · 2011-07-11 18:21:07Z

-1

Use the data.html table in YQL! http://developer.yahoo.com/yql/console

answered Jul 11, 2011 at 18:21

Tilo Mitra

2,7831 gold badge27 silver badges20 bronze badges

1 Comment

user839403 Over a year ago

sorry, i do not find Yahoo! website useful for Indian Companies

Collectives™ on Stack Overflow

Scrape a website (javascript website) using php

3 Answers 3

5 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related