How to import Apache access log into MySQL table?

Question

What is the recommended approach to import Apache access log into a MySQL table? I am thinking of a ruby/perl script + a cron job. It'll be great if there is an example or reference.

I am not sure how to handle the bookmarking of the last log entry in last import either. Suggestions are welcome. Thanks!

Answer 1

Do you own the customlog format? Or do you have to live with whatever is there? if you own the format you can easily delimit the fields (with tabs for example) and its pretty trivial to import.

LogFormat "%{%Y-%m-%d %H:%M:%S}t\t%s\t%h\t%{ClientIP}i\t%U%q\t%{Host}i\t%B\tc1w1402@dc1app14\t%m\t%{User-Agent}i\t%{Referer}i\tDRH\t%T\t%f\t%{Via}i" ezimport

Answer 2

If you can write PERL script then try out mod_perl to log directly to database... for the reference check link .

Answer 3

LogFormat "yourFormat" yourLog

<VirtualHost example.com>
    ServerName example.com
    CustomLog /yourPath yourLog
</VirtualHost>

From here