Import profile data into generic analysis and visualization tool

I have a mechanism for generating profile data of a couple of different types. Are there any tools that will import profile data from external sources and produce useful interactive visualizations? It seems like there are many profiling tools out there that all implement their own rendering engines, but there really isn't much difference between the raw data that is driving them.

So far, I just mimicked gprof's text output, which is more useful than people give it credit for. But that isn't saying much.

Examples: xperf for Windows, Shark for Mac (though I haven't used it), SpeedTracer for Google Chrome, oprofileui, and many others.

gprof2dot is an example of what I'm looking for, in that it accepts output from multiple profile data generators, but (1) I don't find its output terribly useful, especially on large profiles; and (2) optimally, I would like an interactive web-based UI.

Better yet, is there a standard format for profile data that I can use to feed multiple tools?

To start with, I have basic dynamic callgraph data with wallclock timings and call counts, but I would also like to substitute CPU counters, cache miss counts, page fault counts, etc. for the timing information. I would also like to mix together profile data from multiple sources (eg separate profile data from C++ and script profiling tools, related via timestamps or explicit markers.)

Update: I looked a bit further into SpeedTracer. It defines an import/export format for its data. So if nothing else, I can serialize into that format and load my data into SpeedTracer. I'm not crazy about being bonded to Chrome and GWT, but it's a good start.

That is very much the sort of thing I'm talking about, although I would like something already tailored to performance analysis. An OLAP tool would make a fine backend, but I'm looking for more of an existing frontent to reuse.

sfink 2010-07-08 06:05:51

I'd be surprised if you find anything which is a general-purpose front-end. It sounds like something which there isn't much demand for, and I'd expect where there is demand, scripting something custom isn't too much of a burden to justify a reasonable market for such tooling.

codekaizen 2010-07-08 06:41:50

Huh? Not much demand? This would be useful to any project that cares about performance and is willing to define an application-aware set of significant performance events. Anyone who adds dtrace probes is certainly in that category, but that's just getting started. And if all you want is gprof-like output, then sure, it's a small matter of scripting. But to be truly useful, you want interactive waterfall charts, time charts, filterable call graphs, heat maps, etc. Those require more than a little bit of custom scripting.

sfink 2010-07-08 07:13:59

I'm not saying there isn't demand, I'm saying there's not enough to justify the expense of building a standard for, and then retrofit all perf tools to conform to.

codekaizen 2010-07-08 07:17:50

Just reread your comment and noticed the "...justify a reasonable market...." Ah yes, perhaps I should have mentioned that I'm looking for something open source so I can hack it into something that fits my specific application. I agree that it's a poor fit for a proprietary solution. Bundling the data generation and the analysis makes far more sense in that world.

sfink 2010-07-08 07:20:21

ansaurus

tags:

views:

answers:

Import profile data into generic analysis and visualization tool

related questions