ansaurus

Question

How to use PHP to delete X number of lines from the beginning of a text file.

Answer 1

+1 A:

Typical operating systems don't provide the capability to insert or delete content of a file "in-place". What you will need to do is write a function that reads the first file, and creates a new output file containing the lines you want to keep. Then when you're done, delete the old file and rename the new one to the old name.

In pseudocode:

open original file IN for reading
create new output file OUT
read the first two lines from IN
write these lines to OUT
for each line to skip:
    read a line from IN
for the remainder of the file:
    read a line from IN
    write the line to OUT
close IN
close OUT
delete IN
rename OUT to IN

The advantage of this method over some of the other ones presented is that it doesn't require you to read the whole file into memory first. You didn't mention how large your upper size limit was, but if it's something like 100 MB you might find that loading the file into memory is not an acceptable use of space.

Greg Hewgill 2008-10-19 02:22:11

Answer 2

A:

If you can run a linux command, try split. It allows you to split by line count to make things easy.

Otherwise, I'm thinking you'll have to read it in and write to 2 other files.

Darryl Hein 2008-10-19 02:23:01

Answer 3

A:

Alternatively to @Greg's answer, you can read-in the entire file into an array, skip the first X many entries, then rewrite the array to the file.

As an approach: http://us3.php.net/manual/en/function.file-get-contents.php

$fle = file_get_contents("filename");
// skip X many newlines, overwriting the contents of the string with ""
// http://us3.php.net/manual/en/function.file-put-contents.php
file_put_contents("filename", $fle);

warren 2008-10-19 02:23:33

The problem with this is then, depending on how many lines is written, the dates/times will be out of order. Also, I foresee the file writing to the same lines over and over.

PHLAK 2008-10-19 02:53:59

Yes, that could be an issue (writing the same data repeatedly), depending on how large the file is - this is certainly a brute-force method :) .. why would date/times be out of order? unless during the read in php the file was appended-to, I don't think you would see that

warren 2008-10-19 03:10:03

Answer 4

+4 A:

$x_amount_of_lines = 30;
$log = 'path/to/log.txt';
if (filesize($log) >= $max_size)) {
  $file = file($log);
  $line = $file[0];
  $file = array_splice($file, 2, $x_amount_of_lines);
  $file = array_splice($file, 0, 0, array($line, "\n")); // put the first line back in
  ...
}

edit: with correction from by rcar and saving the first line.

bbxbby 2008-10-19 02:31:56

You'll probably want > $max_size rather than ==

Randy 2008-10-19 02:49:43

Answer 5

+1 A:

You could use the file() function to read the file into an array of lines, then use array_slice() to remove the first X lines.

$X = 100; // Number of lines to remove

$lines = file('log.txt');
$first_line = $lines[0];
$lines = array_slice($lines, $X + 2);
$lines = array_merge(array($first_line, "\n"), $lines);

// Write to file
$file = fopen('log.txt', 'w');
fwrite($file, implode('', $lines));
fclose($file);

yjerem 2008-10-19 02:39:21

Answer 6

+4 A:

Use the SPL, Luke

PHP 5 comes with plenty of iterators goodness :

<?php

$line_to_strip = 5;
$new_file = new SplFileObject('test2.log', 'w');

foreach (new LimitIterator(new SplFileObject('test.log'), $line_to_strip) as $line)
    $new_file->fwrite($line);    

?>

It's cleaner that what you can do while messing with fopen, it does not hold the entire file in memory, only one line at a time, and you can plug it and reuse the pattern anywhere since it's full OO.

e-satis 2008-10-19 12:18:56

What's the “SPL Luke”? ;-)

Konrad Rudolph 2008-10-19 12:19:56

Yeah, you need, "Use the SPL, Luke". Aside from that, good answer. :)

Till 2008-10-19 22:59:15

Answer 7

+1 A:

This is a text-book problem of log files, and I would like to propose another solution.

The problem with the "removing lines at the beginning of files" approach is that adding new lines becomes extremly slow, once it has to remove the first lines for every new lines it's writing.

Normal log file appending only involves writing a few more bytes at the end of the file in the file system (and once in a while it has to allocate a new sector, which results in extensive fragmentation - why log files usually are).

But the big problem here is when you are removing a line in the beginning for every row written. The entire file must first be read into memory and then rewritten resulting in huge ammount of I/O to the harddrive (in comparision). To make matters worse, the "split into PHP array and skip first rows" solutions here are extremly slow due to the nature of PHP arrays. This is not a problem if the log file size limit is very small or if it is written to unoften, but with a lot of writes (as in the case with log files), the same huge operation has to be done a lot of times resulting in major performance drawbacks.

This can be imagined as parking cars on a line with space for 50. Parking the first 50 cars is quick, just drive in behind the car infront and done. But when you come to 50, and the car at the front (beginning of file) must be removed you have to drive the 2'nd car to the 1'st position, 3rd to 2nd and so on, before you can drive in with the last car on the 50'th position. (And this must be repeated for every new car you want to park!)

My suggestion is instead saving to diffrent log files, datewise, and then store a maximum of 30 days back etc. Thus taking advantage of the filesystem, which has already solved this problem perfectly well.

CooPs 2008-10-20 03:38:20

Fantastic idea, but for my particular needs, this is unacceptable. I have a small log file, that probably wont be written to much/often. But I'll keep this in mind for future projects.

PHLAK 2008-10-20 07:04:34

ansaurus

tags:

views:

answers:

How to use PHP to delete X number of lines from the beginning of a text file.

Use the SPL, Luke

related questions