parse log file - using java and plot graph

Question

I am writing a log analyzer program which is supposed to plot a graph on search string against time after parsing log file(s). I am currently clueless about how would I parse and store the log file so that it could be processed to create multiple outputs, now I am parsing the entire log file all the time and process to do this – any suggestions?

Answer 1

Don't.

Use either Apache Chainsaw - http://logging.apache.org/chainsaw/index.html - or Lilith - http://lilith.huxhorn.de/ - to parse and view your log files.

You can preprocess your logfiles easily with Perl.

Answer 2

Probably the easiest thing to do is to write a file for each search string. In the file write each timestamp where the string occurs. Then process these files to create a list of search strings and counts. You can then process these files to produce the graphs you want.

Answer 3

In Java, I would look at JFreeChart for drawing graphs once you've parsed the log.

Answer 4

You could use the Pentaho suite to do this, its open source java with good backing and usage.

1. Use Pentaho Kettle to bulk load into MySql
2. Use Report Designer to pull out the stats into a pretty picture