logs

Simple MOLAP solution

To analyze lots of text logs I did some hackery that looks like this Locally import logs into Access Reprocess Cube link to previous mdb in Analisis Service 2000 (yes it is 2k) Use Excel to visualize Cube (it is not big - up to milions raw entries) My hackery is a succes and more people are demanding an access to my Tool. As you see I ...

Best way to wrap rsync progress in a gui?

I use rsync to synchronize files to Windows clients in a server agnostic way. What methods are available to send the progress of rsync to the parent process for display in a gui progress bar? I imagine two or three choices exist. (1) Watch STDOUT (2) Watch rsync.exe log file, similar to unix tail (3) Watch rsync console output in memory...

grep a file, but show several surrounding lines?

I would like to grep for a string, but show the preceding 5 lines and following 5 lines as well as the matched line. I'm scanning for errors in a logfile, and want to see the context. Any clues for the clueless? ...

Quality Control / Log Monitoring

One of the articles I really enjoyed reading recently was Quality Control by Last.FM. In the spirit of this article, I was wondering if anyone else had favorite monitoring setups for web type applications. Or maybe if you don't believe in Log Monitoring, why? I'm looking for a mix of opinion slash experience here I guess. ...

Looking for algorithm that reverses the sprintf() function output

I am working on a project that requires the parsing of log files. I am looking for an fast algorithm that would take groups messages like this: Input: The temperature at P1 is 35F. The temperature at P1 is 40F. The temperature at P3 is 35F. Logger stopped. Logger started. The temperature at P1 is 40F. and puts out something in th...

Software for Webserver Log Analysis?

Can I get some recommendations (preferably with some reasons) for good log analysis software for Apache 2.2 access log files? I have heard of Webalizer and AWStats, but have never really used any of them, and would like to know: What they can do Why they are useful Interesting uses for them Any and all comments and thoughts are welc...

Automated script to zip IIS logs?

I'd like to write a script/batch that will bunch up my daily IIS logs and zip them up by month. ex080801.log which is in the format of ex*yymmdd*.log ex080801.log - ex080831.log gets zipped up and the log files deleted. The reason we do this is because on a heavy site a log file for one day could be 500mb to 1gb so we zip them up whic...

What is the command to truncate a SQL Server log file?

I need to empty an LDF file before sending to a colleague. How do I force SQL Server to truncate the log? ...

An easy way to diff log files, ignoring the time stamps?

I need to diff two log files but ignore the time stamp part of each line (the first 12 characters to be exact). Is there a good tool, or a clever awk command, that could help me out? ...

Best way to rotate Apache log files

I've got an Apache server that has one access log file that is topping 600MB. This makes it really hard to search the file or parse it. What software or modules for Apache are available that will make a daily copy of my access file to make it more manageable? ...

Reading data from a log file as a separate application is writing to it.

I would like to monitor a log file that is being written to by an application. I want to process the file line by line as, or shortly after, it is written. I have not found a way of detecting that a file has been extended after reaching eof. The code needs to work on Mac and PC, and can be in any language, though I am most familiar with...

Appropriate design pattern for an event log parser?

Working on a project that parses a log of events, and then updates a model based on properties of those events. I've been pretty lazy about "getting it done" and more concerned about upfront optimization, lean code, and proper design patterns. Mostly a self-teaching experiment. I am interested in what patterns more experienced designers ...

How do you visualize logfiles in realtime?

Sometimes it might be useful, but mostly just looking cool or impressive to visualize log files (anything from http requests and to bandwith usage to cups of coffee drunk per day). I know about Visitorville which I think look a bit silly, and then there's gltail. How do you "visualize" your log files in realtime? ...

Best tail (log file visualization) freeware tool ?

I have many programs generating tons of logs in files. What freeware do you know, either for Windows or for multi-platforms (if they are in java for instance), able to tail (display the end of) those files ? A plus would be their capacity to display some line in a certain visual style depending on 'grep' criteria Quick update so far...

ETW - Best Fix/Class-Library for Messy Interface ?

By ETW I mean "Event Tracing for Windows". According to my experimenting one virtue is that while by-design it occasionally fails to record events in Very-Busy conditions it otherwise Just Works, as you might expect from a Kernel feature. ETW is the only game in town if you want per-processor-buffering to avoid cache-vs-multi-thread-log...

What non-web-based tools exist to view IIS logs?

I'm looking for non-web-based tools to view IIS logs. I've been using LogParser but I was hoping for something with an interface. It doesn't need to do any fancy reports or charts -- just a list with some search filters is plenty. I need something lightweight that I can run directly on the web server, preferably without a complex inst...

Adding custom log locations to the OS X console application...

After searching online, the best solution I've found so far is to just make a symbolic link in either "/Library/logs/" or "~/Library/logs/" to get it to show up in the Console application. I'm wondering if it would be possible to add a new directory or log file to the "root" level directly under the "LOG FILES" section in the console. ...

A good IIS Log Viewer for large log files?

Hi, I'm looking for a simple IIS Log Viewer that can open large (e.g. 100 meg) IIS Log files and give me simple filtering and searching tools. (IIS = Microsoft Internet Information Server, obviously) (OK, I know I could use Notepad, but I'd like something that can help me filter out the bits I'm not interested in) ...

Accessing crash logs on iPhones used for ad hoc distribution

When using one's own iPhone for development it's easy enough to access any crash logs via XCode->Organizer->Crash Logs. How does one access crash logs on another person's phone if they don't have it set up for development in XCode, as would likely be the case if you were distributing your app to them via ad hoc distribution for beta tes...

Are there any JSON (or YAML) based "time sheet" or "work log" related formats for recording time spent on tasks?

I just need a simple JSON or YAML (or other) text based format for recording the time I spend on tasks. I prefer to do as much work as possible in my text editor (e text editor) so it is more natural to me to stay in the editor and not switch back and forth to programs like Excel (plus this way I retain portable and "open" data). The ...