PHP prevent timeout by using modules that uses http requests

Assuming you want to use HTTP requests, you have a few options, set a timeout, each time less:

function doTaskWithEnd($uri, $end, $ctx = null) {
    if (!$ctx) { $ctx = stream_context_create(); }
    stream_context_set_option($ctx, "http", "timeout", $end - time());
    $ret = file_get_contents($uri, false, $ctx));
    if ($ret === false) {
        throw new \Exception("Request failed or timed out!");
    }
    return $ret;
}

$end = time() + 100;
$fetched = doTaskWithEnd("http://example.com/some/module/fetch", $end);
$ctx = stream_context_create(["http" => ["method" => "POST", "content" => $fetched]]);
$parsed = doTaskWithEnd("http://example.com/some/module/parsed", $end, $ctx);
$ctx = stream_context_create(["http" => ["method" => "PUT", "content" => $parsed]]);
doTaskWithEnd("http://example.com/some/module/save", $end, $ctx);

Or alternatively, with an non-blocking solution (let's use amphp/amp + amphp/artax for this):

function doTaskWithTimeout($requestPromise, $timeout) {
    $ret = yield \Amp\first($requestPromise, $timeout);
    if ($ret === null) {
        throw new \Exception("Timed out!");
    }
    return $ret;
}

\Amp\execute(function() {
    $end = new \Amp\Pause(100000); /* timeout in ms */

    $client = new \Amp\Artax\Client;
    $fetched = yield from doTaskWithTimeout($client->request("http://example.com/some/module/fetch"));
    $req = (new \Amp\Artax\Request)
        ->setUri("http://example.com/some/module/parsed")
        ->setMethod("POST")
        ->setBody($fetched)
    ;
    $parsed = yield from doTaskWithTimeout($client->request($req), $end);
    $req = (new \Amp\Artax\Request)
        ->setUri("http://example.com/some/module/save")
        ->setMethod("PUT")
        ->setBody($parsed)
    ;
    yield from doTaskWithTimeout($client->request($req), $end);
});

Now, I ask, do you really want to offload to separate requests? Can't we just assume there are now functions fetch(), parse($fetched) and save($parsed)?

In this case it's easy and we just may set up an alarm:

declare(ticks=10); // this declare() line must happen before the first include/require
pcntl_signal(\SIGALRM, function() {
    throw new \Exception("Timed out!");
});
pcntl_alarm(100);

$fetched = fetch();
$parsed = parse($fetched);
save($parsed);

pcntl_alarm(0); // we're done, reset the alarm

Alternatively, the non-blocking solution works too (assuming fetch(), parse($fetched) and save($parsed) properly return Promises and are designed non-blockingly):

\Amp\execute(function() {
    $end = new \Amp\Pause(100000); /* timeout in ms */
    $fetched = yield from doTaskWithTimeout(fetch(), $end);
    $parsed = yield from doTaskWithTimeout(parse($fetched), $end);
    yield from doTaskWithTimeout(save($parsed), $end);
});

If you just want to have a global timeout for different sequential tasks, I'd preferably go with doing it just all in one script with a pcntl_alarm(), alternatively go with the stream context timeout option.

The non-blocking solutions are mainly applicable if you happen to need to do other things at the same time. E.g. if you want to do that fetch+parse+save cycle multiple times, independently from each other cycle.

I think "Chain reaction" is a clue that this approach may be over-complicated...

There may be good reasons for switching to a robust messaging/work-queue system such as RabbitMQ or SQS, especially if you are handling significant load. Messaging queues are invaluable in the appropriate context, but they add a lot of complexity/overhead/code if they are used unnecessarily.

Simplest solution

...but if your only concern is preventing a timeout, I would not make it more complicated than it needs to be; you can easily extend or disable timeouts completely using:

set_time_limit(0); //no time limit, not recommended
set_time_limit(300); //5 mins

Your proposed "chaining" pattern is sensible in principle because it allows you to identify precisely where any faults occur, but you can do this all inside the same request/function rather than relying on the network.

Rather than handling faults in one neat location, this would require two (or more) layers of fault handling: one layer which handles the individual request and another that makes the request.

Assuming the work can successfully be handled in a single request (or even with no remote requests at all) then No, it is not a "good solution to use a modular approach with new http requests" because you are adding unnecessary work & complexity by making unnecessary http calls/responses." i.e. This introduces additional failure possibilities, particularly network connectivity/delays, DNS, difficulty testing & debugging, etc.

Separating into separate remote calls may even add 10x network/server/authentication latency and makes it trickier to do sensible things like database connection pooling.

Other ways to simplify the problem?

If possible, it may be worth investigating why this chain of requests takes so long - if you can optimize them to run faster you may be able to avoid adding unnecessary complexity in this part of your system. e.g. Things like database latency or not using db connection pooling could add up to serious overhead across 10 separate processes.

This answer assumes you're using PHP and running tasks by making HTTP requests to each of the URLs in your question.

Your solution depends on what your business requirements are. If you don't care about the order of completion for the HTTP requests, I suggest looking at curl_multi_init() to start learning about the cURL PHP extension's curl_multi_* functions.

If you do care about order of completion (e.g., a specific task must complete before the next), take a look at the curl_init().

To eliminate the possibility of your calling script of timing out, please read about the set_time_limit function or consider forking your process using pcntl_fork.

Alternatively, I would research a message queue. Specifically, check out Amazon's SQS and read about about to interface with it in PHP. Here are a few links regarding SQS and PHP:

• http://docs.aws.amazon.com/aws-sdk-php/v2/guide/service-sqs.html
• http://george.webb.uno/posts/aws-simple-queue-service-php-sdk
• Valid Architecture for a Message Queue & Worker System in PHP?

background jobs with workers is best way because :

Applications often need to perform operations that are time (or computationally) intensive, but it is usually not desirable to do so during a request, as the resulting slowness is perceived directly by the application’s users. Instead, any task that takes longer than a few dozen milliseconds, such as image processing, the sending of email, or any kind of background synchronization, should be carried out as a background task. Furthermore, a worker queue also makes it easy to perform scheduled jobs, as the same queue infrastructure can be utilized by a clock process.

used php resque for implement background jobs : php resque workers