Can I get some recommendations (preferably with some reasons) for good log analysis software for Apache 2.2 access log files?
I have heard of Webalizer and AWStats, but have never really used any of them, and would like to know:
Any and all comments and thoughts are welcome.
I use Analog because it's free, but it is rather dated now (wow, last release was 4 years ago!) and I'm sure it doesn't have as many fancy features as newer ones.
Splunk > is great, and free. It allows you to visualize and search all of your logs in real time. And it's all web based, so you can view your logs from just about anywhere.
It seems that as javascript page-tagging becomes the more popular choice as a way of processing web stats, there's not as much work being done on log-based analysis tools anymore in the marketplace. My office used to use a product called LiveStats.XSP. It wasn't the greatest tool by any means, but it did have some nice features. It was recently bought by Microsoft and is no longer supported however. It abandoned log analysis turned into a proposed Google Analytics killer called Microsoft Gatineau, which supposedly is good at determining the demographics of your visitors, including age and gender (yeah sure...)
When I was looking for log analysis software a while ago, I wanted to avoid anything that looked overly bloated and enterprisey, which is what most stuff seemed to be, focusing more on the marketing and advertising aspects of reports.
One thing you may want to look at is the new version of Urchin, Urchin6 (see features here). Urchin I believe was bought by Google a few years ago. It's offered as a locally installed solution, and with it you have the option to use either page-tagging or log file analysis for any site that it monitors. There also seems to be some interface ties between Google's own web-based Google Analytics and Urchin. It's not free though, unfortunately, and I think you can only get it through authorized partners.
It does all the standard logfile analysis stuff, everything is browser-based, the reports it offers are pretty deep and comprehensive, and it also seems to have a few bells and whistles that other services don't offer. For example, I remember it being able to present a view of a web page it tracks with colored hot spots overlayed on top of it, based on how often users click on items on that area of the page. Worth checking out the demo of it anyways.
AWStats and Webalizer are both good and free (I think both free speech as well as free beer). I generally prefer the look of AWStats - it has a nice modern look whereas Webalizer looks like something created in about 1992.
They both give roughly the same information which includes:
All of this information is usually viewable on a hour by hour, day by day, month by month and year by year basis. Normally the raw data is available but also with bar charts and pie charts. Both AWStats and Webalizer will (I think) try and work out where your visitors come from by using services such as GeoIP, although I never bothered to set this up. Some also try to work out what order people have visited pages in and things like that - but that is very difficult to do so the results are guesses at the best.
I generally find them both useful - even if just to get an overview of what is going on with my server and who is accessing it. They are both relatively easy to install - although I seem to remember Webalizer being a little easier than AWStats, and they both have varied configuration options to let you decide exactly what you want to get out of them.
For more information see their sites at awstats.sourceforge.net/ and http://www.webalizer.org/.
Hope that helps.
Robin
If you are looking for stats that are meant to be given to a customer and provide them with realistic data (not number of hits based on every file), then check out Google Analytics. Free and looks pretty.
I typically do a search every 30 or so days for new projects for better stats. I currently (still) use AWStats (and JAWStats as I mentioned in another thread).
One of the best things about AWStats when compared to it counterparts is ability to break up your datafiles to create reports by day, week, month, year or even hour using the databasebreak feature.
Why people are still using Webalizer and Analog (well, speed aside) is beyond me. At least in my experience they tend to bury as much information as they expose and I end up reading raw logs.
I get Awstats and Webalizer with my web hosting account and I find that neither is accurate or very useful. The reported numbers are inflated by up to 1000%, because the tools don't properly identify bots and spiders. Here is a comparison of the Visits metric between three tools over the past 3 weeks (I think Awstats has only partial data for 3/23, and no data for today 3/24, which is why I did not include the numbers).
Google Woopra Awstats
Sunday, March 1, 2009 10 11 69
Monday, March 2, 2009 13 14 85
Tuesday, March 3, 2009 13 14 96
Wednesday, March 4, 2009 21 28 91
Thursday, March 5, 2009 19 25 107
Friday, March 6, 2009 12 10 88
Saturday, March 7, 2009 12 14 100
Sunday, March 8, 2009 10 11 65
Monday, March 9, 2009 13 14 78
Tuesday, March 10, 2009 17 13 96
Wednesday, March 11, 2009 18 16 87
Thursday, March 12, 2009 19 18 87
Friday, March 13, 2009 12 13 66
Saturday, March 14, 2009 11 7 52
Sunday, March 15, 2009 11 12 57
Monday, March 16, 2009 13 15 92
Tuesday, March 17, 2009 24 22 102
Wednesday, March 18, 2009 18 16 79
Thursday, March 19, 2009 17 18 73
Friday, March 20, 2009 16 11 70
Saturday, March 21, 2009 24 26 67
Sunday, March 22, 2009 103 114 216
Monday, March 23, 2009 232 223 117
I personally prefer Woopra over Google. While it is still in Beta, it can take a long time for your site to get approved, and it will probably be a paid service at some point, the real-time monitoring capabilities are amazing. The new custom reporting capabilities on Google Analytics are superior to Woopra, though. Woopra does not have any capabilties to produce printed reports
I'd go for awstats. I was getting my site usage reports from awstats for a while and then turned on Google Analytics to do this job as well. And, surprisingly, Google turned out to be rather unreliable. I get usually have no visits from a country like Finland and once I knew for sure a friend of mine visited the site from Finland and what happend? Awstats says - hey, you had a visit from Finland! And Google Analytics? I move my mouse over Finland and the bubble says "visits: 0". After that I could never quite make myself trust it again.
Try out XpoLog log analysis for web servers it provide log analysis platform to anlyze multiple web and application servers for centrelized location and create automatic reports and monitors for applications health. find it at XpoLog log analysis