PHP Foreach Loop Speed

Question

I'm trying to take a list of 40,000 items and run a json request for each item. The process is super slow which I suspect is because each json request is finishing before the next request starts.

$result = mysql_query("SELECT * FROM $tableName");

while($row = mysql_fetch_array($result))
{
 checkRank($row['Domain']);
}

checkRank is running something to the effect of:

$json = file_get_contents($jsonurl,0,null,null);

I'm thinking I could run 10 checkRanks at a time to speed the process up? Any other ideas / suggestions?

UPDATE:

For example this loop runs through my array in 27 secs.

for ($i=0; $i<=100; $i++) {
checkRank($domains[$i]);
$i++;
checkRank($domains[$i]);
$i++;
checkRank($domains[$i]);
$i++;
checkRank($domains[$i]);
$i++;
checkRank($domains[$i]);
$i++;
echo "checking " . $i . "<br/>";

}

The loop below takes over 40 seconds with the same array.

for ($i=0; $i<=100; $i++) {
checkRank($domains[$i]);
echo "checking " . $i . "<br/>";

}

Is $jsonurl under your control or is it an external service? — Alex Howansky
– Alex Howansky, Commented Sep 11, 2012 at 19:29
@user1647347: Your code is vulnerable and not safe. Please read my recent post about SQL injections, you could find it interesting: stackoverflow.com/questions/11939226/… — Ilia Ross
– Ilia Ross, Commented Sep 11, 2012 at 19:32
$tableName is not a post or get variable..its listed in my config.php and gets included..I don't think sql injection is an issue here — user1647347
– user1647347, Commented Sep 11, 2012 at 19:37

Community · Accepted Answer · 2017-05-23 12:12:14Z

1

Not sure if this will help much because I don't work with PHP but I did find this.

How can one use multi threading in PHP applications

edited May 23, 2017 at 12:12

CommunityBot

11 silver badge

answered Sep 11, 2012 at 19:32

Ccorock

89212 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user1647347 Over a year ago

Looks interesting, I'll have to test it out.

sarsnake · Accepted Answer · 2012-09-11 19:48:29Z

1

Unless there is something else that you haven't mentioned, the best way to do this would be to do one JSON request and pass your items to it, and get the equal number of results back. That way, you minimize the server response time. I am not sure if you want to send 40,000 items in though, you might want to divide it into 2 parts, but that you can test this later.

so you checkRank() would look something like

function  checkRank($domainsArray) {
$json = file_get_contents($jsonurl,$domainsArray);
}

edited Sep 11, 2012 at 19:48

answered Sep 11, 2012 at 19:31

sarsnake

28k62 gold badges185 silver badges295 bronze badges

6 Comments

user1647347 Over a year ago

Not sure exactly what you mean? Isn't that what I'm doing now? Can you elaborate please?

sarsnake Over a year ago

Right now you are calling a web service (or a server page) 40,000 times which makes your call very slow. I would suggest modifying your server page that you call using JSON to return multiple results, not one.

user1647347 Over a year ago

I dont have control over that. The jsonurl is compete.com API I wrote the same script using ajax and jquery and it was much faster because I performed 10 ajax requests at once. I was hoping to be able to do the same thing here.

sarsnake Over a year ago

well 10 requests is not 40,000. Consider changing it since your scenario changed quantitatively speaking.

user1647347 Over a year ago

no, I'm not saying make 10 requests. Make 10 at a time instead of 1 at a time.

|

user1647347 · Accepted Answer · 2012-09-11 21:54:02Z

1

http://www.phpied.com/simultaneuos-http-requests-in-php-with-curl/

This seems to be a nice way to speed up the processing. Thanks for all the input guys.

answered Sep 11, 2012 at 21:54

user1647347

5071 gold badge8 silver badges24 bronze badges

Comments

Joe Watkins · Accepted Answer · 2012-09-13 18:30:02Z

1

... it's a shame, I read on Stack Overflow today that PHP could not support threading as it would require fundamental changes to the language ...

https://github.com/krakjoe/pthreads

answered Sep 13, 2012 at 18:30

Joe Watkins

17.2k5 gold badges45 silver badges62 bronze badges

Collectives™ on Stack Overflow

PHP Foreach Loop Speed

4 Answers 4

1 Comment

6 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

6 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related