## logfile-analysis

### Software for Webserver Log Analysis?

Can I get some recommendations (preferably with some reasons) for good log analysis software for Apache 2.2 access log files? I have heard of Webalizer and AWStats, but have never really used any of them, and would like to know: What they can do Why they are useful Interesting uses for them Any and all comments and thoughts are welc...

### Perfmon File Analysis Tools

I have a bunch of perfmon files that have captured information over a period of time. Whats the best tool to crunch this information? Idealy I'd like to be able to see avg stats per hour for the object counters that have been monitored. ...

### Tool to put HTTP log file into database (with defined schema)?

So, I want to put the past year or so of Tomcat Combined Format files into a database. There are zillions and zillions of hits. (The plan is to run bespoke and ad hoc queries against it, and match up with some other data. We have some questions that existing log analysis products out there cannot answer for us.) What I am looking for...

### How is MapReduce a good method to analyse http server logs ?

Hello, I've been looking at MapReduce for a while, and it seems to be a very good way to implement fault-tolerant distributed computing. I read a lot of papers and articles on that topic, installed Hadoop on an array of virtual machines, and did some very interesting tests. I really think I understand the Map and Reduce steps. But here...

### How to read log-files over network real fast ?

I'm using Delphi 2007 and have an application that read log-files from several places over an internal network and display exceptions. Those directories sometimes contains thousands of log-files. The application have an option to read only log-files from the n latest days bit it can also be in any datetime range. The problem is that the...

### extracting error information from rails log files

i am developing on 5 different rails projects, plus also refactoring some (moving from older rails versions to 2.3) - what is the best way to extract the error information from the logfiles, so i can see all the depreciation warnings, runtime errors and so on, so i can work on improving the codebase? are there any services or libraries ...

### LogParser query to grab only external IP addresses from IIS logs?

I tried this question over at ServerFault and didn't have any luck, so I thought I'd try here. I'm working on a public website that is used by both external visitors and internal employees. I'm after the external visitor hits, but I can't think of a good way to filter out the internal IP ranges. Using LogParser, what is the best way t...

### What do the square brackets in LaTeX logs mean?

I'm currently working on a parser that reads complete LaTeX logs. Most of the log format is, though weird, easy to figure out, but these square brackets are puzzling me. Here's an example from near the end of one of my logs: Overfull \hbox (10.88788pt too wide) in paragraph at lines 40--40 []$[]$ [] [102]) [103] Kapitel 14. (./Thermo...

### How do I handle large amounts of logfile data for display in dynamic charts?

I have a lot of logfile data that I want to display dynamic graphs from, for basically arbitrary time periods, optionally filtered or aggregated by different columns (that I could pregenerate). I'm wondering about the best way to store the data in a database and access it for displaying charts, when: the time resolution should be varia...

### Exception text in crash log?

I have received a crash log from a beta tester. The crash log was only partially symbolicated (don't know why), but it is caused by an exception: 9 libobjc.A.dylib 0x000048cc objc_exception_throw + 64 10 CoreFoundation 0x0009be8a +[NSException raise:format:arguments:] + 62 11 CoreFoundation ...

### Web log file analysis software to measure search crawlers

I need to analyze the search engine crawling going on in my site. Is there a good tool for this? I've tried AWStats and Sawmill. But both of those give me very limited insight into the crawling. I need to know information like how many unique/distinct webpages in a section of my site was crawled by a specific crawler within a time period...

### Searching huge log files...

Troubleshooting, analyzing & filtering log files is by far one of the most laborious daily jobs. My issue is searching through a log file, which could be well over 4 gigs in size. Simply loading the file takes up to 15 mins. I'm running a fairly fast processor with 8 gigs of memory. After the file loads, I literally only have the luxury ...