Excel VBA - Parse Server Logs

Question

heya,

We have a small project that involves automatically parsing some server/mail logs (among other things). Now, Plan A was to just hack together some Python and Django to do this properly grins, but I got veto-ed and the solution has to be pure-Excel, as it's believed that will be more portable.

1. Importing tab-separated file

Our input file is a "CSV" file, but it's actually a tab-separated file.

Something like:

"Server Name"   "Server Alias"  "1Feb09"    "2Feb09"    "3Fe09" "4Feb09"
"BobsServer"    "Foobar foobar" "34234" "23432" "52234" "23432"
"JanesServer"   "Foobar foobar" "30984" "34233" "34523" "92342"
"SamsServer"    "Foobar foobar" "12321" "23423" "23423" "23423"

etc.

I'm using the following VBA to import this:

Workbooks.OpenText Filename:="C:\logs.csv", Origin:=437, StartRow:=1, DataType:=xlDelimited, TextQualifier:=xlDoubleQuote, ConsecutiveDelimiter:=False, Tab:=True, Semicolon:=False, Comma:=False, Space:=False, Other:=False, FieldInfo:=Array(1, 1), TrailingMinusNumbers:=True

The funny thing is, when the file has a ".csv" extension, it seems to get mangled, and Excel shoves everything into one column. Yet, when the file extension is something else (e.g. ".txt"), it imports fine. It's the exact same file each time Is this some quirk of how Excel handles file extensions and I've missed something?

I'm hoping I don't need to do some hackery in VBA to rename the file each time, I suppose I can, but I'm actually curious why this is happening?

2. Pasting into existing worksheet

Second question, the above opens the CSV file as a whole new sheet, is there a way to instead do the above parsing, but insert it into a temporary new worksheet in our workbook?

3. Read entries, add to master table

The next step is to take our log file, and integrate that into our master worksheet, which contains histories for each server.

From my rather limited Excel knowledge, I was assuming that we'd do something like:

1. Outer loop: Loop through our input log, line by line.
2. Grab the servername, and do a vlookup for it in the master list.
3. If it doesn't exist, we create a new row for this server.
4. Inner loop: Loop through each column of our log.
5. Assuming the server is there now, we go back and grab the date in the log. We then do a hlookup in the master list for our date. We assume it doesn't exist, however, if it does, we can just overwrite that cell. If it doesn't exist, we need to find the right insertion date (there's a chance there might be missing dates in the sequence) and insert our date.

Does the above logic seem sound? Or is there a more efficient way to do it in Excel? (I suspect there probably is, I don't know the Excel object-model very well).

4. Sum up total across arbitary date range

The final step is to actually calculate totals for the servers we want, based on an arbitrary date range. This is just going to be a macro function, that takes three arguments (server, start date, end date).

There's a list of servers we're specifically interested in, I'm guessing we'll probably store this in a separate worksheet. Then we'll just do a fucntion call for each one in that list, and do a horizontal sum across the row, between the two dates. Is there some kind of shortcut I can use here for this part?

Cheers, Victor

Answer 1

Have you considered using Log Parser?

Log parser is a powerful, versatile tool that provides universal query access to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows® operating system such as the Event Log, the Registry, the file system, and Active Directory®.