What applications are there that I can pass data as it's generated and have it analyze some statistics for?

Question

The basic requirement is pass to some command type and execution time (possibly other data as well, but that's the basic data we're concerned with at the moment) from C# code (either managed code or something that can take data periodically from the command line. and perform some statistical analysis on it: avg time for each command type, standard deviation, some charts would be nice, etc.

Something that can do this in real time might be preferable, but I guess it's also acceptable to save the data ourselves and just pass it in to be analyzed.

We could write up something for this, but it seems like there should probably be something out there for this.

Edit: Basically looking for low learning curve and able to do what's mentioned above. Basically something that would be faster to learn and use than coding it manually.

Answer 1

Have you looked at the R project? May be a place to start.

Answer 2

I may be off base here, but would custom Windows perfmon objects and counters do this? You could create an object for each command type, with a counter for execution time, then use Perfmon's logging, charting and reporting facilities. Or export the Perfmon data to Excel/Access for fancier stuff.

Answer 3

Here is an article that explains how to compute variance (standard deviation) and mean as a stream of numbers are coming in.

Answer 4

Here is more on computing statistics on a stream of data as it comes in, drawing from John D. Cook's excellent article: http://stackoverflow.com/questions/282600/#283167