In Java, howd do I iterate through lines in a textfile from back to front

Question

Basically I need to take a text file such as :

Fred
Bernie
Henry

and be able to read them from the file in the order of

Henry
Bernie
Fred

The actual file I'm reading from is >30MB and it would be a less than perfect solution to read the whole file, split it into an array, reverse the array and then go from there. It takes way too long. My specific goal is to find the first occurrence of a string (in this case it's "InitGame") and then return the position beginning of the beginning of that line.

I did something like this in python before. My method was to seek to the end of the file - 1024, then read lines until I get to the end, then seek another 1024 from my previous starting point and, by using tell(), I would stop when I got to the previous starting point. So I would read those blocks backwards from the end of the file until I found the text I was looking for.

So far, I'm having a heck of a time doing this in Java. Any help would be greatly appreciated and if you live near Baltimore it may even end up with you getting some fresh baked cookies.

Thanks!

More info:

I need to search backwards because the file I am reading is a logfile for a game that I host a server for (it's the |err| server on urban terror. check it out). The log file records every event that happens in the game and then my program will parse each event, process it and then act on it (for example, it keeps track of headshots for people and also will automatically kick people who are being d-bags). I need to search back to the most recent InitGame entry so that I can instantiate all the player objects and take care of whatever else needed to be taken care of since the beginning of that game. There are hundreds of InitGame events in the file, but I want the last one. If there is a better way of doing this that doesn't require searching backwards, please let me know.

Thanks

Answer 1

You can just repeat your Python solution using RandomAccessFile and may be a custom subclass of LineNumberReader (or just Reader) on top of it.

Answer 2

Linux has some great text parsing tools that may be better suited than trying to do it in Java.

Answer 3

On searching backwards, two answers come to mind. The first is to search forwards, and keep the last-found InitGame text around for the moment when you reach the end of the file (and overwrite it whenever another InitGame comes along as you are reading the file).

The second solution is to find out the file-size (using f.length()), divide that into large chunks that overlap by more than the maximum size of an InitGame snippet (to avoid problems due to splitting two chunks right on the interesting part), and start reading from the last one and progressing towards the file start (using a Reader's skip() function to jump to your desired reading position: no actual file-splitting is necessary). If you are sure that there are no funny multi-byte chars, RandomAccessFile can be useful.

The most efficient solution, of course, is to read the log-file output as it comes out, keeping a reference to the the last-found InitGame. That way you will never have to re-read the same data twice. You can even set things up so that your java program wakes up once every few seconds, looks at the file, and reads in the newly-added lines.

Answer 4

So, TIL that I need to be more verbose when I explain exactly what I'm doing. Basically I am writing a program that manages a game server that I run. In order for the program to be in sync with the game it needs to find the most recent InitGame line and then read from there so that it can record all this hits, kills, connects and disconnects that it needs to from the beginning of the round. Since a logfile can be quite huge (the last time I forgot to clean one up it was more than 500MB of text), rather than searching from the front, I want to search from the back. In Java there was no built-in way to do this. After searching over a good amount of the internets, I came upon this: http://mattfleming.com/node/11. From that I took out the BackwardsFileInputStream class and used that. Then in my application, I reverse the chars. Next time I should be able to construct my own method, now that I see how it's done and have a better understanding.

So, once the program has read the logfile from the most recent InitGame, it will mimic tail -f and read the logfile as it is written.