Can I get some recommendations (preferably with some reasons) for good log analysis software for Apache 2.2 access log files? I have heard of Webalizer and AWStats, but have never really used any of them, and would like to know: What they can do Why they are useful Interesting uses for them Any and all comments and thoughts are welc...
I have a bunch of perfmon files that have captured information over a period of time. Whats the best tool to crunch this information? Idealy I'd like to be able to see avg stats per hour for the object counters that have been monitored. ...
So, I want to put the past year or so of Tomcat Combined Format files into a database. There are zillions and zillions of hits. (The plan is to run bespoke and ad hoc queries against it, and match up with some other data. We have some questions that existing log analysis products out there cannot answer for us.) What I am looking for...
Hello, I've been looking at MapReduce for a while, and it seems to be a very good way to implement fault-tolerant distributed computing. I read a lot of papers and articles on that topic, installed Hadoop on an array of virtual machines, and did some very interesting tests. I really think I understand the Map and Reduce steps. But here...
I'm using Delphi 2007 and have an application that read log-files from several places over an internal network and display exceptions. Those directories sometimes contains thousands of log-files. The application have an option to read only log-files from the n latest days bit it can also be in any datetime range. The problem is that the...
i am developing on 5 different rails projects, plus also refactoring some (moving from older rails versions to 2.3) - what is the best way to extract the error information from the logfiles, so i can see all the depreciation warnings, runtime errors and so on, so i can work on improving the codebase? are there any services or libraries ...
I tried this question over at ServerFault and didn't have any luck, so I thought I'd try here. I'm working on a public website that is used by both external visitors and internal employees. I'm after the external visitor hits, but I can't think of a good way to filter out the internal IP ranges. Using LogParser, what is the best way t...
I'm currently working on a parser that reads complete LaTeX logs. Most of the log format is, though weird, easy to figure out, but these square brackets are puzzling me. Here's an example from near the end of one of my logs: Overfull \hbox (10.88788pt too wide) in paragraph at lines 40--40 []$[]$ [] [102]) [103] Kapitel 14. (./Thermo...
I have a lot of logfile data that I want to display dynamic graphs from, for basically arbitrary time periods, optionally filtered or aggregated by different columns (that I could pregenerate). I'm wondering about the best way to store the data in a database and access it for displaying charts, when: the time resolution should be varia...
I have received a crash log from a beta tester. The crash log was only partially symbolicated (don't know why), but it is caused by an exception: 9 libobjc.A.dylib 0x000048cc objc_exception_throw + 64 10 CoreFoundation 0x0009be8a +[NSException raise:format:arguments:] + 62 11 CoreFoundation ...
I need to analyze the search engine crawling going on in my site. Is there a good tool for this? I've tried AWStats and Sawmill. But both of those give me very limited insight into the crawling. I need to know information like how many unique/distinct webpages in a section of my site was crawled by a specific crawler within a time period...
Troubleshooting, analyzing & filtering log files is by far one of the most laborious daily jobs. My issue is searching through a log file, which could be well over 4 gigs in size. Simply loading the file takes up to 15 mins. I'm running a fairly fast processor with 8 gigs of memory. After the file loads, I literally only have the luxury ...