Logparser (Microsoft's one) or similar for Unix?

Question

I've been looking for quite some time for an application that fills the same role as Logparser, an awesome piece of technology, but for Unix. Does anyone know of something this? (I've looked at Splunk but its an overkill, a simple command line is all I really need)

Note: Being able to make SQL queries on random logs, is great and much more efficient than grepping and its kin (because you can apply SQL based relational logic to the filtering) and SQL is much more legible than Grep for maintenance purposes when handing off a project to other teams.

Answer 1

The problem with Linux is there isnt really a standard 'log' format. I'm not so sure I have ever seen a log like the ones mentioned in the example.

I think you're better off building a foundation of parsing the log(s) you're interested in based on awk and grep.

Looking at the example they show

SELECT TimeGenerated, SourceName, 
EventCategoryName, Message INTO report.txt FROM Security WHERE 
EventID = 528 AND SID LIKE '%TESTUSER%'

Could be accomplished by select statement:

cat logfile | awk '{print $(1), $(2), $(3)}'

For reference: the awk statment prints columns 1, 2, and 3 respectively.

Refinement step where clause:

./base.sh | grep '528' > report.txt

I think with a little bit of finesse and not this contrived answer, you could come up with something suitable quickly.

Answer 2

There are a couple that come to mind.

1. yaala
2. asql
3. select

yaala support more log file types, and has a sql like query language.

asql only support Apache's log format, and has a simple sql query language.

select support many log formats, and has a sql query language.