Using file() incrementally?

Question

I'm not sure if this is possible, I've been googling for a solution... But, essentially, I have a very large file, the lines of which I want to store in an array. Thus, I'm using file(), but is there a way to do that in batches? So that every,say, 100 lines it creates, it "pauses"?

I think there's likely to be something I can do with a foreach loop or something, but I'm not sure that I'm thinking about it the right way...

Like

$i=0;
$j=0;
$throttle=100;

foreach($files as $k => $v) {
  if($i < $j+$throttle && $i > $j) {
     $lines[] = file($v);
     //Do some other stuff, like importing into a db
  }
  $i++;
  $j++;
}

But, I think that won't really work because $i & $j will always be equal... Anyway, feeling muddled... Can someone help me think a lil' clearer?

Answer 1

You never incremented $i or $j... What you can do, is something like:

    $data = array();
    $chunk = 100;
    $f = fopen($file, 'r');
    while (!feof($f)) {
        for ($i = 0; $i < $chunk; $i++) {
            $tmp = fgets($f);
            if ($tmp !== false) {
                $data[] = $tmp;
            } else {
                //No more data, break out of the inner loop
                break;
            }
        }
        //Process your data


        $data = array();
    }
    fclose($f);

Answer 2

Read the file in line by line for however many lines you need, appending each line to an array. When the array gets to the desired length, process it, and empty the array. E.g.:

$handle = @fopen("/tmp/inputfile.txt", "r");
$throttle = 100;
$data = array();
if ($handle) {
    while(!feof($handle)) {
        $buffer = fgets($handle, 4096);
        $data[] = $buffer;
        if(count($data) == $throttle) {
            doSomething($data);
            $data = array();
        }
    }
    fclose($handle);
}

Answer 3

If by "pause", you mean that you really want to pause execution of your script, use sleep or some of its variants: http://php.net/manual/en/function.sleep.php