2

Making a large database insert, some 8000 rows.

Structure is:

index.html
    <myform></myform>

$('myform').submit(    $.post('upload.php', data, callback(){ alert("success") }');

Given that it's such a huge insert operation, I set

set_time_limit(300);

Script runs for some 120 seconds, inserting about 5000 rows and then stops.

At this point it downloads upload.php, with no content. Doesn't throw an error, PHP error log is empty.

Don't know where to begin trying to troubleshoot this?


Sample

7/1/2014   20:05:25.6 [GV1     ] ATR_21A_COM_FLT_F              CFN             FAULT      ATR COMM FAULT   

Javascript

$('#upload-form').submit(function (evt) {

    evt.preventDefault();
    var payload = {
        'alarms' = $('#upload-text').val(),
    };

    $.post("upload.php", payload, function (data) {
        if (data == "success") {
            alert("Success");
        } else {
            alert("Failure");
        }
    });
});

PHP

<?php

set_time_limit(300);

$alarms = $_REQUEST['alarms'];

/********** Explode alarms by line break **********/
$alarms = explode("\r\n", $alarms);

/********** Array to hold values for database **********/
$temp = [
    "datetime"    => "",
    "location"    => "",
    "label"       => "",
    "remainder"   => "",
];

/********** Connect to MySQL **********/
$link = new mysqli('localhost', 'USER', 'PASS', 'DBASE');
if ($link->connect_errno > 0) {
    die("<p>Error connecting to MySQL</p>");
}


foreach ($alarms as $key => $row) {

    $row = trim($row);

    /********** Unset frivelous alarms and continue loop **********/
    $check_regex = ('/(CFN|FAULT|OK)/');
    if (preg_match($check_regex, $row) == 0) {
        unset($alarms[$key]);
        continue;
    }

    /********** Explode space delineation **********/
    $temp_row = explode(" ", $row);

    /********** Unset empty elements and reindex **********/
    $temp_row = array_filter($temp_row);
    $temp_row = array_values($temp_row);

    /********** Convert date/time to datetime **********/
    $temp_date = date("Y-m-d", strtotime($temp_row[0]));
    $temp_time = date("H:i:s", strtotime($temp_row[1])); 
    $temp['datetime'] = trim($temp_date) . " " . trim($temp_time);

    /********** Unset empty elements and reindex **********/
    unset($temp_row[0], $temp_row[1]);
    $temp_row = array_filter($temp_row);
    $temp_row = array_values($temp_row);

    /********** Preg remove [*], not needed **********/
    $temp_row = implode(" ", $temp_row);
    $regex = "/\[.*?\]/";
    $temp_row = preg_replace($regex, " ", $temp_row);

    /********** Explode space delineation **********/
    $temp_row = explode(" ", $temp_row);

    /********** Reindex **********/
    $temp_row = array_filter($temp_row);
    $temp_row = array_values($temp_row);

    /********** Get alarm location **********/
    $temp['location'] = $temp_row[0];

    /********** Unset empty elements and reindex **********/
    unset($temp_row[0]);
    $temp_row = array_filter($temp_row);
    $temp_row = array_values($temp_row);    

    /********** Unset rows that are not faults **********/
    $eject_regex = "/(CFN|FAULT)/";
    if (preg_match($eject_regex, $temp_row[0]) != 1) {
        unset($alarms[$key]);
        continue;
    }

    /********** Unset empty elements and reindex **********/
    unset($temp_row[0], $temp_row[1]);
    $temp_row = array_filter($temp_row);
    $temp_row = array_values($temp_row);    

    /********** Get Alarm Label **********/
    $temp['label'] = implode(" ", $temp_row);

    $insert = $link->prepare("INSERT INTO alarms (`alarm_timestamp`, `alarm_location`, `alarm_label`) VALUES (?, ?, ?)");
    $insert->bind_param('sss', $temp['datetime'], $temp['location'], $temp['label']);

    if ($insert->execute()) {
        $insert->free_result();
    } else {
        die($link->error);
    }
} 

echo "success";

I know it's really ugly code. I care more about building the database at this point, that's the finished product I'm going to show to get the thumbs up to move forward and write clean code.

5
  • For troubleshooting post the code you are trying. Commented Jul 15, 2014 at 3:34
  • You may not actually be overriding your script execution time limit with that call... Commented Jul 15, 2014 at 3:35
  • scrowler, it's running on localhost. And when it does time out without that line it's in the php error log. Commented Jul 15, 2014 at 3:38
  • 1
    It is hard to see what you're doing. Where is the DB insert? A 8000 row insertion isn't really "huge", and shouldn't take a whole minute to run unless you've got a bunch of crazy business going on with cascading triggers. Commented Jul 15, 2014 at 3:41
  • looks like your client (browser) is aborting the operation. I agree with @Peter 8k rows it not huge, normally you may only require less than the default time out of 30 secs. You may need to check the insertion instead of trying to set huge timeouts. Your client may not respect the delay at the server. Commented Jul 15, 2014 at 3:55

1 Answer 1

2

By process of elimination and looking at the components involved, and keeping in mind that this is all (obviously) very tentative:

  • PHP: It's most likely not a PHP-related problem (max_execution_time directive, etc.) since you're running on localhost, and have no errors showing up in your PHP log.
  • MySQL: It's most likely not a MySQL related problem as records are getting correctly inserted up to a point.
  • Browser: It's most likely not a client-related problem because if the browser would decide to cancel the operation, you would probably see a browser error instead of your result page.
  • Web server: So, that leaves the web server. While both Apache and IIS use a default timeout setting of 300 seconds, it might me overridden in the configuration. If you happen to be using Zend Server on Windows, it seems that the default request timeout there is 120 seconds, which matches your problem description very well.
Sign up to request clarification or add additional context in comments.

1 Comment

Am in fact using Zend. I'll check it in the morning and get back. Cheers.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.