How to develop a real-time event analytics solution?

Question

Hello all.

I need to develop an in-house real-time event analytics solution for our web application (e.g: mixpanel, chartbeat, kissmetrics like tool). We mainly use ruby on rails, Redis, MySql and JavaScript.

The analytics solution must have an API and must be real-time. I've never done any API nor worked with real-time data.

What technologies should I use in order to accomplish my goal?

Regards, Donald

Answer 1

I've done this kind of project earlier.

1. For the graphs I'd recommend Google Visualization API

2. For storage I would recommend a Round Robin scheme. There is a Round Robin database engine called RRDtool, but you can also implement a Task (or table partition) in your MySQL database that would speed queries once the stored data gets too big to be queried fast enough, by deleting (or separating) old data.