How to read a range of rows from CSV file to JSON array using PHP to handle large CSV file?

Question

The target is how to read a range of rows/lines from large CSV file into a JSON array in order to handle large files and read the data in pagination method, each page fetches a range of lines ( e.x. page number 1 fetch from line 1 to 10, page number 2 fetch from line 11 to line 20, and so and ).

the below PHP script read from the being CSV file to the desired line ($desired_line), My question is how we can determine the starting line to read from a specific line ($starting_line)

<?php
// php function to convert csv to json format
function csvToJson($fname, $starting_line, $desired_line) {
    // open csv file
    if (!($fp = fopen($fname, 'r'))) {
        die("Can't open file...");
    }
    
    //read csv headers
    $key = fgetcsv($fp,"1024","\t");
    
    $line_counter = 0; 
 
    
    // parse csv rows into array
    $json = array();
        while (($row = fgetcsv($fp,"1024","\t")) && ($line_counter < $desired_line)) {
        $json[] = array_combine($key, $row);
        $line_counter++; 
    }
    
    // release file handle
    fclose($fp);
    
    // encode array to json
    return json_encode($json);
}



// Define the path to CSV file
$csv = 'file.csv';
print_r(csvToJson($csv, 20, 30));

?>

Well if the condition ($line_counter < $desired_line) applies an upper limit, then what the counterpart for that would be, to apply a lower limit "in the other direction", should be pretty obvious, is it not? — C3roe
– C3roe, Commented Jan 14, 2022 at 13:54
Incorporate PHP Generator to generate line by line directly to JSON string: PHP - json_encode a generator object (using yield) — Justinas
– Justinas, Commented Jan 14, 2022 at 13:55
You have $starting_line and $desired_line there. How about you start with figuring out how to pass them as arguments to your function; then tweak your condition. And then do a bit of research as to whether it's possible to read specific lines only instead of looping until you get there. Imagine the wasted cycles with that approach in start 10000, until 10100... — Markus AO
– Markus AO, Commented Jan 14, 2022 at 14:30
@MarkusAO >> For pass the starting_line and desired_line as arguments, I updated the function — Sam.
– Sam., Commented Jan 14, 2022 at 17:16

Flash Thunder · Accepted Answer · 2022-01-14 17:18:24Z

0

You should use functions like:

fgets() to read the file line by line

fseek() to move to the position of the last fgets() of the chunk

ftell() to read the position for fseek()

Something like this (it's only a schema):

<?php

...

$line_counter = 0;
$last_pos = ...
$fseek($fp,$last_pos);
while($line = fgets($fp)){ // read a line of the file
   $line_counter++;
   (...) // parse line of csv here
   if($line_counter == 100){
      $lastpos = ftell($fp);
      (...) // save the $lastpos for next reading cycle
      break;
   }
}

...

?>

You can also skip the fseek() and ftell() part and just count the lines every time from the beginning, but that will generally have to go through the whole file from the beginning till the desired lines.

edited Jan 14, 2022 at 17:18

answered Jan 14, 2022 at 17:13

Flash Thunder

12.1k10 gold badges55 silver badges103 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to read a range of rows from CSV file to JSON array using PHP to handle large CSV file?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related